-- Logs begin at Wed 2022-07-20 08:55:59 UTC, end at Wed 2022-07-20 09:53:34 UTC. -- Jul 20 09:21:25.153052 dv-cinder-822822-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 09:21:25.161600 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 09:21:25 2022] *** Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: nodename: dv-cinder-822822-3-iscsi Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: machine: x86_64 Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: clock source: unix Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: pcre jit disabled Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: detected number of CPU cores: 4 Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: current working directory: / Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: detected binary path: /usr/bin/uwsgi-core Jul 20 09:21:25.163609 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: your processes number limit is 47680 Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: your memory page size is 4096 bytes Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: detected max file descriptor number: 2048 Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: lock engine: pthread robust mutexes Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: thunder lock: enabled Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 09:21:25.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 09:21:25.208269 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: Python main interpreter initialized at 0x561a3c9be9c0 Jul 20 09:21:25.208269 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: python threads support enabled Jul 20 09:21:25.208269 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: your server socket listen backlog is limited to 100 connections Jul 20 09:21:25.208269 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: your mercy for graceful operations on workers is 90 seconds Jul 20 09:21:25.208654 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 09:21:25.208724 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: *** Operational MODE: preforking *** Jul 20 09:21:25.208966 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: *** uWSGI is running in multiple interpreter mode *** Jul 20 09:21:25.208966 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: spawned uWSGI master process (pid: 80173) Jul 20 09:21:25.209174 dv-cinder-822822-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 09:21:25.211320 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: spawned uWSGI worker 1 (pid: 80184, cores: 1) Jul 20 09:21:25.211320 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: spawned uWSGI worker 2 (pid: 80185, cores: 1) Jul 20 09:21:25.211484 dv-cinder-822822-3-iscsi devstack@keystone.service[80173]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 09:21:30.659842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 09:21:30.659842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: from cryptography.utils import int_from_bytes Jul 20 09:21:30.660602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 09:21:30.660602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: from cryptography.utils import int_from_bytes Jul 20 09:21:30.661792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 09:21:30.661792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: from cryptography.utils import int_from_bytes Jul 20 09:21:30.662233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 09:21:30.662233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: from cryptography.utils import int_from_bytes Jul 20 09:21:30.709305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 09:21:30.709468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 09:21:30.709676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] command line args: [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 09:21:30.709775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 09:21:30.709926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 09:21:30.710014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 09:21:30.710085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ================================================================================ {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 09:21:30.710147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] command line args: [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 09:21:30.710299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] admin_token = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.710384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 09:21:30.710515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ================================================================================ {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 09:21:30.710592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] config_dir = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.710666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.710726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] admin_token = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.710874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] config_source = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.710959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] config_dir = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.711021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] debug = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.711180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.711267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] config_source = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] debug = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_config_append = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.714964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_dir = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_file = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_config_append = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.715932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.716140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.716140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_dir = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.716531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.716698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_file = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.716887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.717121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.717322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.717322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.717681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.717872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.718865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.719235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.719235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.719631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.719826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.720689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] publish_errors = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.721953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] publish_errors = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.722993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] standard_threads = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.723859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] use_journal = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] standard_threads = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] use_json = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] use_stderr = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] use_syslog = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.724944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.725071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.725203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.725203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] use_journal = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.725441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.725578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] use_json = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.725780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.725914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] use_stderr = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.726040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.726166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] use_syslog = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.726166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 09:21:30.726418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.726418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.726675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.726808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.726937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.727072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.727199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.727352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.728992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.729913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.730711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.731996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.732897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.733799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.734948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.735889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.736938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.737949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.738988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.739115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.739242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.739418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.739560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.740826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.740999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.741191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.741415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.741646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.741850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.742014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.742205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.742394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.742590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.742849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.743113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.743336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.743541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.743821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.744991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.745185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.745375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.745612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.745929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.746892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.747891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.748857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.749972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.750768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.751940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.752996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.753351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.753351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.753674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.753674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.753925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.754987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.755956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.756900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.external = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.757931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.password = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] auth.token = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.758998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.external = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.759955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.password = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] auth.token = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.760899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.761939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.762947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.763968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.764889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.765959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.766886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.767926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.768859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.769877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.770934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.771891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.772919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.773963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.774092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.774221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.774861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.775949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.776874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.777942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.778998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.779977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.780911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.781961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.782883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.783950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.784886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.785932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.786980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.787952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.788885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.789811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.790006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.790686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.790827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.790959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.791896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.792933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.793968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.794896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.795955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] role.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] role.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.796962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.797878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] role.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.798930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] role.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.799905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.800954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.801875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.802908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.803862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.804925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.805910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.806976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.807927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.808978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.809880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.connection = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.810955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.811939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.812864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.connection = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.813936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.814862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80185) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.815106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.816204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.816204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.816204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 09:21:30.816204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80184) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 09:21:30.826870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.827101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.827347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.829014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.829244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.829466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.835681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.838735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.867513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.867754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.868614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:30.869004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.013413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.017370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.063372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.063693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.064115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.064349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.129611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.129831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.130976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.131261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.131561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.131846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.156648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.157971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.158103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.158275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.158558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.158840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.159122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.169700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.176337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.182805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.183026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.183244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.183491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.183703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.183914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.184124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80184) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.187954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.188288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.188612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.188930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.189245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.189560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.189887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80185) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 09:21:31.201738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.201854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.201934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 09:21:31.202211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 09:21:31.202294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.202443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 09:21:31.202709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.202837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.202962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.203202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 09:21:31.203311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.203479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.203736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.204085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.204431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.204772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.215279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.215568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.215833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.216109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.217156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.217528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.217905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.218280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.224219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.226650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.227552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.227972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.228234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.228622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.228900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.229158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.229423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.229690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.229954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.230223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.230483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.231128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.231527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.231898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.232376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.232754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.233109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.233482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.233848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.234209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.234573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.234932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.257106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.259241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.259533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.262487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.264280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.264625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.264798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.265076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.265349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.270291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.270683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.271058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.272622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.272900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.278040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.278319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.278591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.278863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.279129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.279416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.279679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.279902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.279989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.280339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.285080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.285464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.285839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.286233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.286600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.286969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.287352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.287711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.298722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.298991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.303560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.307213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.307520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.307736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.308113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.312521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.312920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.316131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.316805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.317201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.319884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.320176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.320452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.320734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.321001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.321794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.323526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.327268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.331150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.331569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.331970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.332355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.332742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.333143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.340767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.341133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.341397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.341656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.349563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.351274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.351562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.353141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.353619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.353991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.354378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.354707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.354984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.355258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.355656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.355932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.362926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.364764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.365147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.369038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.369433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.369828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.369916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.370189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.370329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.370472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.370746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.370831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.371020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.371318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.371590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.384912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.385297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.385679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.386059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.386441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.386830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.387220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.390812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.394086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.397427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.399027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.401479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.401752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.402032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.408269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.409497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.411109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.411932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.414410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.414692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.415607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.415878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.416420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.418146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.419929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.420313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.420944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.425353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.425634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.425905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.428852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.430617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.436156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80184) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.436281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.437236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.437236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.437236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.437236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80184) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.445616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.445932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.446228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.454199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80185) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 09:21:31.454479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.454755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.455021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.455309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.455578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.455853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.456123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.456396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.456660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.456936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.457207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.457452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.457718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80185) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 09:21:31.475453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561a3c9be9c0 pid: 80184 (default app) Jul 20 09:21:31.479369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57fe2-8942-40be-83bd-6cfc0e207123 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:31.479660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57fe2-8942-40be-83bd-6cfc0e207123 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:31.479934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a57fe2-8942-40be-83bd-6cfc0e207123 None None] PATH_INFO: `/v3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:31.485808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Jul 20 09:21:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:31.497327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561a3c9be9c0 pid: 80185 (default app) Jul 20 09:21:39.122958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:39.123560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:39.123656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:39.124940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:21:39.132337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:39.132756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:39.132901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:39.204677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: INFO dbcounter [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] Registered counter for database keystone Jul 20 09:21:39.226594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG oslo_db.sqlalchemy.engines [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80185) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 09:21:39.233737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writer thread running {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 09:21:39.411009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG passlib.handlers.bcrypt [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80185) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 09:21:39.411379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG passlib.handlers.bcrypt [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80185) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 09:21:39.467303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:39.468938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:39.590112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 09:21:39.783196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:39.853392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: INFO dbcounter [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] Registered counter for database keystone Jul 20 09:21:39.872922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG oslo_db.sqlalchemy.engines [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80184) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 09:21:39.879053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writer thread running {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 09:21:39.998648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2eb05f-5bf0-4c37-a877-6488776cb56a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:40.002956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:40.003644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:40.003932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:40.004212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:40.012599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:40.064242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 09:21:40.064531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1f3b5-b781-41f2-a8f7-bf2fab3bb60b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:40.074561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Jul 20 09:21:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:21:40.080575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:40.098049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163f9ce-8392-4413-b56c-9c1a04c19a7e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:40.102530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:40.103669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:40.104028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:40.104311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] PATH_INFO: `/v3/projects/c93172815ff14813957f03bf548f6155` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:40.105795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] RBAC: Authorizing `identity:get_project(project_id=c93172815ff14813957f03bf548f6155)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:40.156559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 09:21:40.157110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65e393-0989-42e0-a0ba-e71d0bceae4c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:40.161858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Jul 20 09:21:40 2022] GET /identity/v3/projects/c93172815ff14813957f03bf548f6155 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:42.201837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fce6a-6bef-489e-8639-208fb0186651 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:42.202374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fce6a-6bef-489e-8639-208fb0186651 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:42.202465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fce6a-6bef-489e-8639-208fb0186651 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:42.206551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:21:42.211985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:42.212417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:42.212542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:42.232905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:42.234427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:42.242760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 09:21:42.264988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fce6a-6bef-489e-8639-208fb0186651 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:42.276919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fce6a-6bef-489e-8639-208fb0186651 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:42.281102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:42.281733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:42.282018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:42.282292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:42.290584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:42.292390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c4069-fda8-4e48-a08a-75467c52a05f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:42.301349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Jul 20 09:21:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:21:42.307736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:42.316775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f5631-70fa-43ac-a56e-59fac2aaf84b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:42.321366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:42.322228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:42.322514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:42.322794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:42.324247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:42.326062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed94a4-c9fe-4416-b142-3c03bb8bfcac admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:42.327669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Jul 20 09:21:42 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:44.399136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e28e9-e25c-4c59-98aa-2eb453f7be5e None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.399703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e28e9-e25c-4c59-98aa-2eb453f7be5e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.399703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e28e9-e25c-4c59-98aa-2eb453f7be5e None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.401067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:44.406107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.406553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.406648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.423659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:44.424844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:44.466019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8144cf-3641-470a-850d-70ad3a2f99d7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.466309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8144cf-3641-470a-850d-70ad3a2f99d7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.466589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8144cf-3641-470a-850d-70ad3a2f99d7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.467990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:21:44.473121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.473414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.473697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.508308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:44.514190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG passlib.handlers.bcrypt [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80184) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 09:21:44.514678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG passlib.handlers.bcrypt [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80184) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 09:21:44.532596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:44.556090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:44.560721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:44.562306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:44.636199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5b094-04e8-4136-8cf2-a1f7780994b9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:44.662821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:21:44.670933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:44.671760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.672049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.672327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.683481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:44.685245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:44.694745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:44.696437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:21:44.697282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 6/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:21:44.702286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:44.782148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:44.786738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463b82-9e35-41db-90a6-60dcad432a39 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:44.789303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:44.800596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75450d7-9076-4632-a81f-a4fa5c067ee5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:44.824261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:44.825165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.825455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.825733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.834003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:44.834748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.835035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.835325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.836133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:44.836432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:44.837831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:44.849218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 7/15] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:21:44.854736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:44.891633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:44.893151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:44.952501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004afe4-70e6-4664-bc7d-a37acdc28843 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:44.985364 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:44.986233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:44.986512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:44.986790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:44.988035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:44.989492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:44.990165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 9/16] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:21:44.992605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 8/17] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:21:44.996866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:44.996979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:45.000082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:45.012411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:45.095907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c528e4-e84a-4fcd-afe8-40e08f10d6f7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:45.102139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320794e-5bb4-4257-850a-e73ecc52af0c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:45.129432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:45.130219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:45.130500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:45.130768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:45.132154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:45.134559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:45.135853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:45.136194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:45.136655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:45.136938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:45.137215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:45.141812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:45.147783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:45.149294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:21:45.167591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:45.168612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:45.169552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:21:45.169552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:21:45.169552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:21:45.169552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:21:45.169552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:21:45.174775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:45.213032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:45.236601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:45.245517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68938-c375-4105-8f69-8d8ce23f8b2c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:45.278980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7f965-9cbc-457a-b004-be4ef1bd4e3e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:45.281605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:45.282281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:45.282572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:45.282850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:45.283717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.domain.id=default, user.id=cee8d1e7eb98438287c0fee8c54bc3cc) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:45.284029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:45.285490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27972eb9-4164-45f7-9c7a-5a16d7620b9a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:45.295750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Jul 20 09:21:45 2022] GET /identity/v3/role_assignments?user.id=cee8d1e7eb98438287c0fee8c54bc3cc&scope.domain.id=default&role.id=743f7cb41c634460b2981782f093ad19 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:21:45.319068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:45.319791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:45.320110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:45.320388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:45.321135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:45.321443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:45.322885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225115-8744-48e0-a10a-7797b2e6c740 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:45.342389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:21:47.416816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec3fa-11db-47ff-bc5a-1eb60249d4cf None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:47.417355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec3fa-11db-47ff-bc5a-1eb60249d4cf None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:47.417355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec3fa-11db-47ff-bc5a-1eb60249d4cf None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:47.418899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:47.435252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:47.435756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:47.435826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:47.465234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:47.466519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:47.538334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 11/23] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:47.561205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec3fa-11db-47ff-bc5a-1eb60249d4cf None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:47.673682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec3fa-11db-47ff-bc5a-1eb60249d4cf None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:47.709454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:47.710301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:47.711163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:47.711474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:47.715251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:47.724822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-4581d800-98d4-4c20-a462-129def3dbb21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:47.736775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:47.737817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:21:47.738788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 13/24] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:21:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:47.744301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:47.775078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766b6-c326-40e2-becb-55927d71f20f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:47.775388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766b6-c326-40e2-becb-55927d71f20f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:47.775662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291766b6-c326-40e2-becb-55927d71f20f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:47.777100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 14/25] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:21:47.787588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:47.787870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:47.788147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:47.821279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:47.822481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:47.858521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:47.958360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3be57d-a310-4970-bef8-aa8a25fa6d45 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.005059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 15/26] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 09:21:48.031588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.032445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.032731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.033005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.033774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:48.034081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.035522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.074849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 12/27] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:48.080985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.088384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.269812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae282-eda4-4e44-a4b7-5547b5d00dde None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.277890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475efd3d-c1b8-41dd-aea4-f791c18e0fdc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.311647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:48.313915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.315223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.315682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.317319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.322248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.325659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:48.327466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bc717-9a79-49d9-8412-10e5cb49273a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.330086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 16/28] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:48 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:48.332107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3064-6bce-4ba9-b46b-045c86002579 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.336373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.337622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.338020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3064-6bce-4ba9-b46b-045c86002579 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.338819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3064-6bce-4ba9-b46b-045c86002579 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.340343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 17/29] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:48.340792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.341215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.342663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3064-6bce-4ba9-b46b-045c86002579 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.365622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.367612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-59d55057-10b4-4892-9d3b-c5c9da97ced0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:21:48.399352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:48.400233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:48.401037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:21:48.401037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:21:48.401037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:21:48.401037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:21:48.402933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 13/30] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:21:48.404685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.405196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.405679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.435337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:48.437337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:48.530246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3064-6bce-4ba9-b46b-045c86002579 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.566771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.567849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.568284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.568694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.570158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.571785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.573554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 18/31] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:48.574492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.608764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 14/32] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:48.609675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.640825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:48.686318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e2208-84f6-4a9b-8f97-2052eb7161d6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.727333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.727772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.728275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.728763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.729683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:48.730197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.731807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.755548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 19/33] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:48.756829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:48.766292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:48.774830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c1dcb-a25a-481d-ac63-f40e9858c8ba None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.825204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.827036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.827821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.828304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] PATH_INFO: `/v3/domains/default/users/cee8d1e7eb98438287c0fee8c54bc3cc/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.831501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cee8d1e7eb98438287c0fee8c54bc3cc, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.833175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a4a58-1d6f-40da-96bb-a7bb8d3ee871 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.872447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845d9eb-fdec-4c9c-b3d1-4c8fc75c5982 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:48.902184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:48.915873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:48.916691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.917072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.917479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.918372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:48.920284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e9088d-2e9c-4a7c-ab41-fa2758a911a4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:48.965866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 15/34] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Jul 20 09:21:48 2022] PUT /identity/v3/domains/default/users/cee8d1e7eb98438287c0fee8c54bc3cc/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:21:48.967949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86448a36-9abc-428b-b4d7-6872dde51a77 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:48.968390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86448a36-9abc-428b-b4d7-6872dde51a77 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:48.968809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86448a36-9abc-428b-b4d7-6872dde51a77 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:48.978730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 16/35] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:48.980143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86448a36-9abc-428b-b4d7-6872dde51a77 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:49.068686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 20/36] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jul 20 09:21:48 2022] POST /identity/v3/roles => generated 219 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:49.072108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa92431-c047-4733-9dc2-382339001365 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.072627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa92431-c047-4733-9dc2-382339001365 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.073129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa92431-c047-4733-9dc2-382339001365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.094953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8aa92431-c047-4733-9dc2-382339001365 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:49.097107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8aa92431-c047-4733-9dc2-382339001365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:49.191539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 21/37] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:49.234312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa92431-c047-4733-9dc2-382339001365 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:49.240384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86448a36-9abc-428b-b4d7-6872dde51a77 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:49.298081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:49.298685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.298970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.299355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.300090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:49.301785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a33e8-c222-47ce-afbe-d8cc67965b2e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:49.342557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 17/38] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:21:48 2022] POST /identity/v3/projects => generated 301 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:49.344013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c108-390d-4638-a69b-d63381f5f81f None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.344339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c108-390d-4638-a69b-d63381f5f81f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.344679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c108-390d-4638-a69b-d63381f5f81f None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.346152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 18/39] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:49.352278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.352618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.353052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.400685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa92431-c047-4733-9dc2-382339001365 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:49.420171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:49.464797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:49.465511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.465797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.466078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.466885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:49.468733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cacd9-38ed-4687-9798-6b2c6bb387f3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:49.487326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 22/40] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/roles => generated 225 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:49.488841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ed9a-a673-44b7-b5b8-78e112ac4cce None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.489279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ed9a-a673-44b7-b5b8-78e112ac4cce None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.489565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1ed9a-a673-44b7-b5b8-78e112ac4cce None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.491039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 23/41] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:49.513499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.513842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.514125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.528573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:49.530235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:49.534141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:49.541696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:49.542952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:49.633252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 19/42] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:21:49.644636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 24/43] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:49.663117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:49.666753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:49.799707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756135d4-3edd-41a6-849c-7675d2569b09 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:49.801278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:49.804077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283360c9-07fb-481b-873d-0b3438c8e0bc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:49.836901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:49.837583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.837874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.838152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.838906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:49.840382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd86d05-c6cb-4764-9fa9-1e80a48f6e2f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:49.845436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:49.846521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:49.846893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:49.847273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:49.848900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:49.850590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:49.853298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 20/44] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:49.858807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:49.859864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 25/45] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/roles => generated 223 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:49.872075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:49.973756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588dfd-1c52-4c13-b477-67e6de2919bc None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:50.011979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:50.012651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:50.012932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:50.013215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:50.013931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:50.015433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c520c-0e82-4f89-9bca-53c7cbff80eb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:50.042074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 21/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:21:49 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:21:51.288022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5e3f-87d6-4818-872b-aae97b171b97 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:51.289240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5e3f-87d6-4818-872b-aae97b171b97 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:51.289240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5e3f-87d6-4818-872b-aae97b171b97 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:51.290657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 26/47] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:51.296099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:51.296531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:51.296670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:51.314314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:51.315530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:51.371671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:51.384935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 22/48] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:51.407838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef5e3f-87d6-4818-872b-aae97b171b97 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:51.510910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef5e3f-87d6-4818-872b-aae97b171b97 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:51.543365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:51.544095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:51.544379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:51.544651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:51.554742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:51.562461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:51.563363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:21:51.564137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 27/49] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:51.571489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:51.670104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55452e36-5e7b-4d86-8328-435dc70a9f61 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:51.704014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:51.704683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:51.704965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:51.705239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:51.705981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:51.706287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:51.707715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:51.715817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 23/50] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:51.721369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:51.746060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:51.821152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8f7e-a1c8-43c8-9720-3e6951348c59 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:51.853947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:51.854828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:51.855116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:51.855428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:51.856638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:51.858090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:51.859768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 28/51] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:51 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:51.865174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:51.919136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:51.966123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af1484-5715-4237-a99b-1898c0e3d73e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:51.999803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:52.000594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.000879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.001159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.002331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:52.003806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a77af-af56-4389-89c2-94a45f4fc327 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:52.005433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 24/52] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:52.011770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:52.092707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89e831-e2cb-4a4a-9db9-0a89d9240b2f None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.092915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89e831-e2cb-4a4a-9db9-0a89d9240b2f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.093193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89e831-e2cb-4a4a-9db9-0a89d9240b2f None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.094574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 25/53] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:21:52.099445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.099732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.100007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.100441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:52.110735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980383f-8dbf-4807-8691-368139e7da35 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:52.116811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:52.118010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:52.145139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:52.145794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.146081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.146359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.147198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.domain.id=default, user.id=cee8d1e7eb98438287c0fee8c54bc3cc) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:52.148278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:52.149742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:52.155536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 29/54] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Jul 20 09:21:52 2022] GET /identity/v3/role_assignments?user.id=cee8d1e7eb98438287c0fee8c54bc3cc&scope.domain.id=default&role.id=743f7cb41c634460b2981782f093ad19 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:52.189977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 26/55] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:21:52.215522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:52.319278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd42496-0a79-4003-8e8c-842e2722d8ce None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:52.353648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:52.354370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.354651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.354923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.356145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:52.357591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc04da-56d0-4cd4-af1e-31272ebfe586 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:52.359430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 30/56] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:52.365037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:52.413919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:52.462577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e5987-d9e5-4143-b51e-138a9b5c703d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:52.495577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:52.496234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:52.496519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:52.496798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:52.497513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:52.498950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4e0ab-b12f-4283-bd5b-f60aa2367f82 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:52.522299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 27/57] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Jul 20 09:21:52 2022] POST /identity/v3/projects => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:54.565024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0689-e2a1-44dd-8eaf-5aa2f28f279b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:54.565550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0689-e2a1-44dd-8eaf-5aa2f28f279b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:54.565640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba0689-e2a1-44dd-8eaf-5aa2f28f279b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:54.566959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 31/58] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:54.573228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:54.573618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:54.573782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:54.591004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:54.592209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:54.656204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 28/59] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:54.681046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba0689-e2a1-44dd-8eaf-5aa2f28f279b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:54.808592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:54.884282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba0689-e2a1-44dd-8eaf-5aa2f28f279b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:54.916288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:54.917075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:54.917364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:54.917647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:54.918831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:54.920556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffc12a-ed16-4306-89af-b3e029990c07 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:54.922175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 32/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:54.927735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:54.958367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:55.024839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec070b-d552-492e-a23c-3124476ec691 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:55.057816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:55.058479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:55.058809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:55.059037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:55.059773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:55.061218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5ec63-8798-4cee-bcc5-a6d0077c0ddd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:55.085144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 29/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:21:54 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:58.284543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999e0ed-c59e-42a6-8110-fc2712f57038 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.284543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999e0ed-c59e-42a6-8110-fc2712f57038 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.284543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999e0ed-c59e-42a6-8110-fc2712f57038 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.284543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 33/62] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:58.290518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80aa2-d0b5-4ca4-a3fb-2575cd04071b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.291280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80aa2-d0b5-4ca4-a3fb-2575cd04071b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.291738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b80aa2-d0b5-4ca4-a3fb-2575cd04071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.315072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d9b80aa2-d0b5-4ca4-a3fb-2575cd04071b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:58.316533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d9b80aa2-d0b5-4ca4-a3fb-2575cd04071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:58.401651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 30/63] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:58.434435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999e0ed-c59e-42a6-8110-fc2712f57038 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:58.467990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffc8a-43d6-43a5-a7d7-1164592f0608 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.468293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffc8a-43d6-43a5-a7d7-1164592f0608 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.468562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ffc8a-43d6-43a5-a7d7-1164592f0608 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.470117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 31/64] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:58.476218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c828-5887-4c21-a502-dc7faac87bea None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.476760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c828-5887-4c21-a502-dc7faac87bea None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.477251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687c828-5887-4c21-a502-dc7faac87bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.505517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2687c828-5887-4c21-a502-dc7faac87bea None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:58.506913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2687c828-5887-4c21-a502-dc7faac87bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:58.542141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:58.620094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 32/65] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:21:58.621833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea651-091c-4a2f-8d44-5950b32974ff None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.622334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea651-091c-4a2f-8d44-5950b32974ff None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.622824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea651-091c-4a2f-8d44-5950b32974ff None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.624401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 33/66] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:58.627639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:58.630164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595f86-d4b6-4777-a102-2f3c6098e894 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.630617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595f86-d4b6-4777-a102-2f3c6098e894 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.631035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595f86-d4b6-4777-a102-2f3c6098e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.640064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999e0ed-c59e-42a6-8110-fc2712f57038 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:58.661367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7b595f86-d4b6-4777-a102-2f3c6098e894 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:58.663644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7b595f86-d4b6-4777-a102-2f3c6098e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:58.690050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:58.690865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.691148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.691477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.709036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:58.720655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:58.721629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 34/67] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:58.722844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:58.744881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 34/68] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:58.746466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f773-91b5-47a3-8bb4-89888fdece52 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.746933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f773-91b5-47a3-8bb4-89888fdece52 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.747402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f773-91b5-47a3-8bb4-89888fdece52 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.748856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 35/69] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:58.750051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f773-91b5-47a3-8bb4-89888fdece52 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:58.853146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:58.859935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142ffd-54c1-425d-b923-3948661d8423 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:58.892847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f773-91b5-47a3-8bb4-89888fdece52 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:58.900610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:58.901416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.901703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.902011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.915386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d11a62c7-b8cb-41a0-8bee-ec4a29351b7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:58.926204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:58.927084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:21:58.927890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 35/70] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:21:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:58.929375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517833-5138-42f2-8430-b62d10d89fbb None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.929552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:58.929698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517833-5138-42f2-8430-b62d10d89fbb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.929935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70517833-5138-42f2-8430-b62d10d89fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.930204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:58.930491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:58.930765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:58.931550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:58.931856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:58.934538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:58.945518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 36/71] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:21:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:58.950290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:58.950396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-70517833-5138-42f2-8430-b62d10d89fbb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:58.950396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-70517833-5138-42f2-8430-b62d10d89fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:58.954167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.028303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 36/72] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:59.037066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfd86-2868-411d-b515-18811d268c23 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.037066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfd86-2868-411d-b515-18811d268c23 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.037066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dfd86-2868-411d-b515-18811d268c23 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.037066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 37/73] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:21:59.037066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dfd86-2868-411d-b515-18811d268c23 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.070820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45395e-aca3-4336-994d-477c633c9c27 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.110502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.111241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.111556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.111839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.119985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.123818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:59.132893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:59.133782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:21:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:59.139655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.149797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dfd86-2868-411d-b515-18811d268c23 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.193855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.194875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.195188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.196565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:59.196872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.198252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cd3ac-54b2-4ab6-90c5-862ff74f2ad7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.215074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 38/75] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.216914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.217360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.217868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.225155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.239457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:59.240660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:59.257247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f3779-15a2-4659-b5f6-2c27c5b01c27 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.300711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.302088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.302551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.303034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.311808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 39/76] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:21:59.312707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.325122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.326679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.328443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 38/77] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.329215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.394995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.425628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2957860-83e1-46dd-94be-d27f10de20c0 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.440784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831d754-1408-4efb-9d85-6ec54ca19b98 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.456844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.462548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.463390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.463677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.463956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.473871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:21:59.475754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.476428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.476716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.476995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.477741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:59.478052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.479476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:21:59.482395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:21:59.483401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:21:59.484227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 40/78] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:21:59.484858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.488277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 39/79] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.488920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.587492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c9e52-aec3-428b-9040-219e50b8c761 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.590887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c1736-9226-4d3d-b6a2-3f74563bb75b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.607180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.621955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.622612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.622895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.623173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.625011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.625891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.626174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.626454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.627711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.629169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.630825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 41/80] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.632560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.690239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.690968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.692608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.732038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134e7e-a8a4-4fd6-9918-72704f3330d0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.765236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.766039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.766326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.766609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.767836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.769294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.771086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 42/81] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.771817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.793614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 40/82] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:21:59.794897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.815885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.870659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30ac8c-f6e3-46b9-b967-3c439c67e16e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.891541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64825677-0c95-4efb-b88c-f49beb971f4b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:59.895395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:59.903922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.904803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.905096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.905380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.906621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.908100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.909745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 43/83] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.910514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:59.926087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:59.926918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:59.927188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:59.927492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:59.928772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:59.929248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:59.930707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:59.938839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 41/84] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:59.939808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.009312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccbe1-706a-4d34-92a9-b4d3c41d5aa1 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.020468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.038913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2dd3-6ff0-4cf3-9edb-dd82d509475e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.042130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.042776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.043060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.043363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.072868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.073544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.073833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.074115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.074833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.076326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.095698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 42/85] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:21:59 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:00.096697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.107590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.117604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.119131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.211943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cde7c8-1b29-4433-a64d-0914c4e25aa8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.250496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 44/86] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:00.251404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.252615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.253562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.253785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.254065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.255270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.257292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.259031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 43/87] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:00.259896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.270161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.366342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.375981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d9943-9001-44f6-ab22-06a0d94f94b5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.398448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc6b8-0dbd-4caf-9f44-3d9dc731eb16 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.426107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.426767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.427049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.428103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.442275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.442958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.443261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.443581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.445585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.447131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.448983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 44/88] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:00.449688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.536834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.541785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.543684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.562950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcc131-4b2e-48d0-be66-74ed4eae44ed None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.598695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.599637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.599926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.600287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.601510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.603037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.604861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 45/89] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:00.610430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.660058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.667271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 45/90] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:00.668048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.742750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f327-70e4-4c70-9f9b-2bb3bc3718d6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.780127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.780790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.781075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.781351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:00.782383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:00.782700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:00.784183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:00.792132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 46/91] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=011a188e9a5b4469804831444edf3f23 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:00.792902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:00.811177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.813442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bba9f-539e-4a41-94f8-fa3b5abbcb52 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:00.886362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:00.887041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:00.887351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:00.887631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.002978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:01.003762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a1b2d-851c-415f-9e8f-f7f9f57c522b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:01.012552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.014059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.043065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:01.043773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:01.044061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:01.044335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.045531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=cee8d1e7eb98438287c0fee8c54bc3cc) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:01.045847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.047328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.054008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/role_assignments?user.id=cee8d1e7eb98438287c0fee8c54bc3cc&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:01.054839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:01.132309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:01.167123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4844cb-e541-42d1-a203-c235c3e3c95b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:01.180228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 46/93] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:01.181457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:01.205023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:01.205987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:01.206275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:01.206552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.207879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.209528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.211775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 48/94] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:01 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:01.217885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:01.306385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:01.317594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad95ec-9173-44d3-9fbf-a5029a08bca8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:01.335075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4036c-b8d6-4ddf-834d-6abea44ba1e8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:01.380250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:01.380930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:01.381213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:01.382059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.382905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:01.383217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.384707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.390660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:01.391626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:01.391909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:01.392229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] PATH_INFO: `/v3/projects/3a55ae21682648f2b1cfea192f281398` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.394245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] RBAC: Authorizing `identity:get_project(project_id=3a55ae21682648f2b1cfea192f281398)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.395761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c945-ae88-474a-abf7-07f231093fca None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.401598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 49/96] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:01 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:01.402101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 47/96] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:01 2022] GET /identity/v3/projects/3a55ae21682648f2b1cfea192f281398 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:01.409947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:01.413329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:01.547602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7f5d6-3ec4-4224-847a-1d1e800c6e65 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:01.587849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:01.588446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:01.588729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:01.589006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:01.589849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3a55ae21682648f2b1cfea192f281398, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:01.590162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:01.591836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a09f3-e3d0-406e-9e01-bdd8b1025dd2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:01.598375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 50/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:01 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=3a55ae21682648f2b1cfea192f281398&role.id=011a188e9a5b4469804831444edf3f23 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:02.585927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f965-401e-4b27-865d-f675d5bae129 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:02.586426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f965-401e-4b27-865d-f675d5bae129 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:02.586865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f965-401e-4b27-865d-f675d5bae129 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:02.589460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 48/98] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:02.607213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431485-36d7-460b-83aa-5e59965d1856 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:02.607719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431485-36d7-460b-83aa-5e59965d1856 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:02.607793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83431485-36d7-460b-83aa-5e59965d1856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:02.632240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-83431485-36d7-460b-83aa-5e59965d1856 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:02.633451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-83431485-36d7-460b-83aa-5e59965d1856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:02.709342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:02.722838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 51/99] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:02.758239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f965-401e-4b27-865d-f675d5bae129 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:02.833089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c8302-a21c-4505-b628-24d57dc1187d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:02.833403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c8302-a21c-4505-b628-24d57dc1187d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:02.833650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c8302-a21c-4505-b628-24d57dc1187d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:02.835874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 52/100] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:02.840967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:02.841751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:02.842695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:02.880350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:02.882030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:02.895744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f965-401e-4b27-865d-f675d5bae129 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:02.963198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:02.964005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:02.964297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:02.964577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:02.966482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:02.967964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:02.969844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 49/101] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:02.989255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:03.000131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 53/102] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:03.044835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:03.058225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:03.195153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c0e2b-6258-48f1-afda-2bc5d72d9254 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:03.263866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:03.264537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:03.264820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:03.265096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:03.265802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:03.267242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:03.271407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a050136-54d5-4e00-b7e9-edf9ffd9a24c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:03.316552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:03.328053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:03.328868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:03.329151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:03.329429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:03.339352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 50/103] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:22:02 2022] POST /identity/v3/projects => generated 302 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:22:03.344719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:03.360887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:03.361807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:03.363027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 54/104] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:03.368508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:03.522761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:03.541481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e026a-b26a-4b7b-bc88-d74a87ef0737 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:03.576360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:03.577069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:03.577358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:03.577634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:03.578379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:03.578685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:03.580103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:03.590296 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 51/105] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:03.595525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:03.719423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebed18d-c910-4bd2-90ca-d5f97ec71834 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:03.762027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:03.762909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:03.763245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:03.763546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] PATH_INFO: `/v3/users/f9734794ff554b359699b4a12127a524` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:03.782162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] RBAC: Authorizing `identity:get_user(user_id=f9734794ff554b359699b4a12127a524)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:03.783593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:03.785309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 55/106] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:03 2022] GET /identity/v3/users/f9734794ff554b359699b4a12127a524 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:03.791543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:03.900057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5213e-bc65-4581-85b0-7424031f77fa None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:03.942637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:03.943307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:03.943624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:03.944462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:03.968991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:04.031397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:04.032931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:04.156351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 52/107] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:04.161894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:04.219013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:04.290479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdd77c-2e0b-4011-8ae7-e063929cd43c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:04.332036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:04.333296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.333798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.334896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.340100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:04.341826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc239e-2916-4e9f-8efd-35b26fb73e68 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:04.344502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 56/108] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:04 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:04.350798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:04.382666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2836f-5be1-45a2-ba8a-4fb7562b2fe5 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.383311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2836f-5be1-45a2-ba8a-4fb7562b2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.383805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2836f-5be1-45a2-ba8a-4fb7562b2fe5 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.386086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 57/109] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:04.392296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.392927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.393681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.417487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:04.419046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:04.423567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:04.513714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 58/110] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:04.541515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:04.549471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4addc70-93a7-44cc-9093-e493085a079f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:04.594387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:04.637901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:04.638601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.638862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.639137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.640013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=f9734794ff554b359699b4a12127a524) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:04.640332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:04.641801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218977b4-b238-4a13-94c6-56b94be23747 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:04.648417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 53/111] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:04 2022] GET /identity/v3/role_assignments?user.id=f9734794ff554b359699b4a12127a524&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=f16da423656e46c386eac96321ea205e => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:04.680739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181a99-ca8f-498d-a236-404e01ae35dc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:04.745665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:04.746486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.746770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.747046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.768427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:04.770359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade7d9-0961-4603-a009-d861a574b19d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.770897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade7d9-0961-4603-a009-d861a574b19d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.771431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ade7d9-0961-4603-a009-d861a574b19d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.773270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 54/112] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:04.777781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:04.778740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:04.779762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 59/113] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:04.782029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e4cb9-5c2a-4d89-879b-1eeb19f2f090 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.782396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e4cb9-5c2a-4d89-879b-1eeb19f2f090 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.782750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e4cb9-5c2a-4d89-879b-1eeb19f2f090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.789387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:04.813446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-612e4cb9-5c2a-4d89-879b-1eeb19f2f090 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:04.814692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-612e4cb9-5c2a-4d89-879b-1eeb19f2f090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:04.910774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 55/114] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:04.916219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01edb8-68af-453c-a162-f2757fbf465a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.916219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01edb8-68af-453c-a162-f2757fbf465a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.916219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01edb8-68af-453c-a162-f2757fbf465a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.916219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 56/115] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:04.916540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a52b0-f002-4d68-97eb-9232e3caef7d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.916612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a52b0-f002-4d68-97eb-9232e3caef7d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.918979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a52b0-f002-4d68-97eb-9232e3caef7d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.918979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 57/116] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:04.924017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:04.924379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:04.924756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:04.944371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:04.945659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:04.962651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d613536-7a5e-491d-914f-e59039925337 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:04.964891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:04.993508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.006882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.008058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.008615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.008897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.009652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:05.009958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.011896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ceb0a-fb9f-48fb-a9b1-45c9692d654c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.022652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 60/117] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:05.024127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.024416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.024699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.037065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 58/118] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:05.038518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.062646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:05.065595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:05.144839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 61/119] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:05.145818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.164178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06976ba7-f0a8-4dc6-9737-16879b63183f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.219533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.220780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.221067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.221343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.235883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:05.245589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:05.246523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:05.247301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:05.250707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 59/120] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:05.257018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.267487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.275390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65779a2-db6e-460a-844e-40a6fcba4725 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.293160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.315937 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.317116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.317605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.317997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.319935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.321741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.328258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 62/121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:05.328258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.383449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e110f1-9220-49a0-b295-a83d8f036d87 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.424035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.424856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.425138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.425413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.430278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3ac5-8570-4230-978e-32cd13593894 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.443314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:05.455826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:05.456732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 60/122] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:05.457517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.473206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.474294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.474574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.475616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.492031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:05.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:05.502734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:05.503536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:05.504855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 63/123] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:05.505641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.541185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.570319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.585520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec52aae-3840-431e-9489-d62e7043af83 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.623181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32a6df-5414-4176-a9bf-40e0c148aa54 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.631426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.632480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.632902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.633350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.634717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:05.635595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.637399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.649547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 61/124] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:05.650990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.666843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.667931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.668357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.668705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.670324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.672256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.674288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 64/125] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:05.675058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.777213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8345a99-772d-4148-9117-c0e904bcb774 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.783310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.797501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4a596-f26a-4b00-8772-4d867e7e56e2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.818151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.819338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.819893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.820269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.821310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:05.821659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.823177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.832371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 62/126] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:05.837454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.844967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:05.845665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:05.846046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:05.846369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:05.847129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:05.847528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:05.849022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:05.857604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:05.858777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 65/127] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:05.859850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:05.964240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9530e-02d2-46a8-8ee3-f770207e81f9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:05.972800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d8231-f7cf-489a-beee-8b5b4f83a80f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.012845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.013732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.014017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.014295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.015934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.017393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.019202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 63/128] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:05 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.020214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.026696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.027667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.027962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.028252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/d3f391bb67354ffe82a26d3d321c1224/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.030295 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, user_id=d3f391bb67354ffe82a26d3d321c1224, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.032099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.049108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 66/129] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:05 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/d3f391bb67354ffe82a26d3d321c1224/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:06.049758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.070054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.162261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.189115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf381845-e826-4ce9-8e65-819634b6ee4c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.200721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca52039-2b39-4609-ae4c-1b03f4fa66d0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.239180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.240099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.240443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.241238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.243680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.245129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.248258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 64/130] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.249873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.264079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.265492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.266095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.266603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.272242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.274120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8943ba-52e1-4223-9fcb-a21e3b0ed933 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.276516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 67/131] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.277998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef178d-95a9-4ad3-8f7e-58a54b637288 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.278487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef178d-95a9-4ad3-8f7e-58a54b637288 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.279209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef178d-95a9-4ad3-8f7e-58a54b637288 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.284866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 68/132] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:06.286032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef178d-95a9-4ad3-8f7e-58a54b637288 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.360612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd28eb-2dc4-4309-9938-31051f2800b9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.383620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.398968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.400315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.400606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.400884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] PATH_INFO: `/v3/projects/3a55ae21682648f2b1cfea192f281398` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.402093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] RBAC: Authorizing `identity:get_project(project_id=3a55ae21682648f2b1cfea192f281398)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.403611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.405530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 65/133] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/projects/3a55ae21682648f2b1cfea192f281398 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.406104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.412044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef178d-95a9-4ad3-8f7e-58a54b637288 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.442641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.465725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.466592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.466877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.467155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.468393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.469875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3f2e5-7052-4fff-a71d-a213836616d4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.476972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 69/134] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.478400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.478597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.478884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.498337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:06.499738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:06.546292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26704c-3dc0-44b4-b0d1-dc28eb74a3ce None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.579916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 70/135] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:06.584268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.590808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.591980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.592938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.593590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.603309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.603309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.603966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 66/136] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.604726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.685529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.704952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:06.711628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9d16a-fcf4-4241-839c-c1b1c6c526d9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.722379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857afd10-8e15-4c22-b6f7-a1ccb16d9f1b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.761729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.762706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.763002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.763272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] PATH_INFO: `/v3/projects/3a55ae21682648f2b1cfea192f281398/users/d3f391bb67354ffe82a26d3d321c1224/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.765152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.765288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] RBAC: Authorizing `identity:create_grant(project_id=3a55ae21682648f2b1cfea192f281398, user_id=d3f391bb67354ffe82a26d3d321c1224, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.766443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.766729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.767168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.767678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/d3f391bb67354ffe82a26d3d321c1224/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.770312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, user_id=d3f391bb67354ffe82a26d3d321c1224, role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.772701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.785105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 71/137] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:06 2022] PUT /identity/v3/projects/3a55ae21682648f2b1cfea192f281398/users/d3f391bb67354ffe82a26d3d321c1224/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:06.786960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.792048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 67/138] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:06 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/d3f391bb67354ffe82a26d3d321c1224/roles/97a9c200717c42a4a09a99543c15fa03 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:06.796656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:06.924578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2f80a-adbf-44d4-a511-4aefc2f5894a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:06.978202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:06.978953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:06.979232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:06.979530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:06.982090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:06.988618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:06.990844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 72/139] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:06.996972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:07.004234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131984-b9b1-4f0b-b49c-047909de080e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:07.019698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:07.050213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:07.051129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.051446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.051902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/cee8d1e7eb98438287c0fee8c54bc3cc/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.054965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, user_id=cee8d1e7eb98438287c0fee8c54bc3cc, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:07.056508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ab7e3-f146-4445-b4c4-1bfaa3212328 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:07.076261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 68/140] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:06 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/cee8d1e7eb98438287c0fee8c54bc3cc/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:07.152600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eab64-1072-4fc7-b69e-d3fe280e6a14 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.152930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eab64-1072-4fc7-b69e-d3fe280e6a14 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.153163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eab64-1072-4fc7-b69e-d3fe280e6a14 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.154582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 69/141] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:07.161111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.161447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.161683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.178493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9923d7-1e68-4b21-8065-af1e62c9abcc None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:07.185626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:07.195861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:07.200804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:07.244204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:07.244819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.245100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.245371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.246113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:07.247623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:07.280828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 73/142] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:06 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:22:07.291671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 70/143] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:07.314977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:07.390106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:07.445617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634db6-2559-4811-b3f5-161600303d30 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:07.486142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:07.487023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.487321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.487600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.500120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:07.508132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:07.509011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:07.509804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 74/144] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:07.515647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:07.629096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194cb79-17e5-4e93-af65-3e4fd40faf0b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:07.665863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:07.666539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.666826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.667103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.667886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:07.668196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:07.669854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:07.672137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:07.681069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 71/145] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:07.687883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:07.820961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d338317-20cf-4f1c-a8f8-336d5b9d1dec None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:07.853120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:07.866745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:07.867728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:07.868016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:07.868336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] PATH_INFO: `/v3/users/f9734794ff554b359699b4a12127a524` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:07.869554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] RBAC: Authorizing `identity:get_user(user_id=f9734794ff554b359699b4a12127a524)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:07.871023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:07.872697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 75/146] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:07 2022] GET /identity/v3/users/f9734794ff554b359699b4a12127a524 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:07.878279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:07.992666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51819569-872d-4a59-8a12-a1f61a8d7fb2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:08.029167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:08.030105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:08.030397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:08.030731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:08.031973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:08.033451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3224314-9744-4e35-a218-16205ad16469 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:08.042615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:08.053310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 72/147] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:07 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:08.154112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed419ddd-6166-4411-936f-1c49c0a37fa3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:08.195029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:08.195976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:08.196257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:08.196531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/f9734794ff554b359699b4a12127a524/roles/f16da423656e46c386eac96321ea205e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:08.198693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, user_id=f9734794ff554b359699b4a12127a524, role_id=f16da423656e46c386eac96321ea205e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:08.200288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8163e7-37f6-4585-aa95-878ef289bd96 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:08.217414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 76/148] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:08 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/users/f9734794ff554b359699b4a12127a524/roles/f16da423656e46c386eac96321ea205e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:08.602196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b72e3-578b-43d4-bb33-b158aa4289d3 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:08.602574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b72e3-578b-43d4-bb33-b158aa4289d3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:08.602773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b72e3-578b-43d4-bb33-b158aa4289d3 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:08.604189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 73/149] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:08.609728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:08.610021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:08.610299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:08.632378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:08.633603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:08.689783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:08.716493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 77/150] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:08.740978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b72e3-578b-43d4-bb33-b158aa4289d3 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:08.815607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:08.854867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b72e3-578b-43d4-bb33-b158aa4289d3 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:08.890246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:08.891268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:08.891575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:08.891857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:08.904287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:08.912516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:08.913428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:08.914280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 74/151] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:08.921336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:09.029984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f34c6-e806-4137-919f-a34dcbf5fabf None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:09.068992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:09.069708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:09.069988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:09.070265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:09.071017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:09.071349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:09.072802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:09.081709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 78/152] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:09.093384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:09.209670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506df54-e2d4-4f3f-b340-82db64a77f24 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:09.239653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:09.250460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:09.251374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:09.251661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:09.251936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:09.253850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:09.255680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:09.258086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 75/153] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:09 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:09.263346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:09.315691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:09.391544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f67cb-d1a8-4c69-aedc-beebdec283d7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:09.427767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:09.428922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:09.429358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:09.429634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:09.530257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:09.531768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:09.588854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:09.649782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 79/154] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:22:09.655281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:09.799741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec46783-e473-4ade-87fc-e98454a65f96 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:09.842870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:09.844100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:09.844581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:09.845050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:09.852033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:09.853717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22bbd7-f03a-4c31-9c9e-31ebebc22215 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:09.855717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 76/155] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:09 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:09.864378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:09.995838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ba3e7-2157-496e-951c-08121e930f40 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:10.030160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:10.030836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:10.031126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:10.031430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:10.036211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:10.036536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:10.038536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3f40e-5eaf-48eb-a681-40fad7295f7a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:10.045289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 80/156] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:09 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=011a188e9a5b4469804831444edf3f23 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:10.839590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85356fae-6df0-40f2-ad48-f76b4aebb40f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:10.840619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85356fae-6df0-40f2-ad48-f76b4aebb40f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:10.841111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85356fae-6df0-40f2-ad48-f76b4aebb40f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:10.842698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 77/157] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:10.848380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef322b-9910-4117-baf3-e43b295339b9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:10.848664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef322b-9910-4117-baf3-e43b295339b9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:10.848932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef322b-9910-4117-baf3-e43b295339b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:10.872019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:10.881151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-47ef322b-9910-4117-baf3-e43b295339b9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:10.882462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-47ef322b-9910-4117-baf3-e43b295339b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:10.971900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 81/158] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:11.003395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85356fae-6df0-40f2-ad48-f76b4aebb40f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:11.100936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:11.146830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85356fae-6df0-40f2-ad48-f76b4aebb40f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:11.182934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4206-2787-4b44-aa19-5afa83336094 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.183905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4206-2787-4b44-aa19-5afa83336094 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.184384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4206-2787-4b44-aa19-5afa83336094 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.187236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 82/159] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:11.193575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.194310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.194819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.228516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:11.229854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.230347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.230815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.232825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:11.234240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:11.260650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:11.289182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:11.290193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:11.290979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:11.291615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 78/160] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:11.300887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:11.348729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 83/161] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:11.392472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:11.502393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:11.527047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920784-a512-43e0-913c-1488843f78d8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:11.567266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:11.576930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8a3a9-2005-427f-a1bb-b6bed778a34c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:11.605763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:11.606447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.606729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.607003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.607784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:11.608088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:11.609457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c68d73-c785-4a69-a527-93201ea60b03 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:11.631845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 79/162] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:11.633202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd5ea-aa82-412a-8bc3-d0691182b4f5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.633490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd5ea-aa82-412a-8bc3-d0691182b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.633764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd5ea-aa82-412a-8bc3-d0691182b4f5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.635217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 80/163] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:11.635711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:11.636524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.636801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.637206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.640096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debd5ea-aa82-412a-8bc3-d0691182b4f5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:11.658280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-548883a9-e40a-4014-9416-be7501e509a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:11.667102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:11.667999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:11.668786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:11.678969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 84/164] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:11.680302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.680589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.680868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.711809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:11.712997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:11.822921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 85/165] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:11.823858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:11.905871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:11.932349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debd5ea-aa82-412a-8bc3-d0691182b4f5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:11.949605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e8cb0-550a-47aa-b01a-39b974e58e1e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:11.984737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:11.985681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:11.986122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:11.986561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:11.987478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:11.987927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:11.989475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.003413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 86/166] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:12.004832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.072410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.074240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.074951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.075643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.077955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.080835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129eb3b6-7b24-46f5-bfbf-99ec9a1cd442 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.092882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 81/167] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:11 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:12.095441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e66b-5b32-4176-82d9-35ff123630d7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.096112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e66b-5b32-4176-82d9-35ff123630d7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.096792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659e66b-5b32-4176-82d9-35ff123630d7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.100082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 82/168] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:12.101543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659e66b-5b32-4176-82d9-35ff123630d7 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.121385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d6f81-782b-4544-ad7a-098630c53b0a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.147897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.164773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.165914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.166983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.167495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.181060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:12.190649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:12.191994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:12.192924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:12.201214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 87/169] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:12.202299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.250860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.322283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d6bf6-0f6b-4008-ae3c-b25931a83666 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.362218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.363225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.363738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.363938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659e66b-5b32-4176-82d9-35ff123630d7 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.364439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.418235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.419116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.419699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.419973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] PATH_INFO: `/v3/users/d3f391bb67354ffe82a26d3d321c1224` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.421180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f391bb67354ffe82a26d3d321c1224)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.422683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425ae54-a2c4-4956-9523-a972cf1496b2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.429123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 83/170] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/users/d3f391bb67354ffe82a26d3d321c1224 => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:12.429123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.429123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.429123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.436306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.448389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:12.449555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:12.461513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.463990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ececb-e703-4750-a6af-8563cec7f29b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.524012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 84/171] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:12.525110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.573365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 88/172] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:12.575317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700283-1825-45e7-943e-7e90e46c974b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.575888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700283-1825-45e7-943e-7e90e46c974b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.576376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3700283-1825-45e7-943e-7e90e46c974b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.577869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 89/173] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:12.578805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3700283-1825-45e7-943e-7e90e46c974b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.582743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.638687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf22849-51ff-4848-8460-fcf364b1bbea None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.667779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.678079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.679006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.679304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.679591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.680574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:12.680752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.682158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.691159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 85/174] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:12.691997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.695171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3700283-1825-45e7-943e-7e90e46c974b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.732683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.733790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.734289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.734778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.797885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ac66-aa5d-4cc8-9725-4fb4e1b8ff95 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:12.810076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.812099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.812385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.836418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:12.837286 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.837566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.837837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] PATH_INFO: `/v3/projects/3a55ae21682648f2b1cfea192f281398` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.839131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] RBAC: Authorizing `identity:get_project(project_id=3a55ae21682648f2b1cfea192f281398)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:12.840650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e03a7-f63c-47a8-aea8-072ddc46a135 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:12.842667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 86/175] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/projects/3a55ae21682648f2b1cfea192f281398 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:12.848471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:12.848761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:12.849036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:12.867097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:12.868415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:12.903249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:12.933169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 90/176] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:12.934543 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:12.944693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 87/177] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:12.945995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.046717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fc2d5-b404-4b03-a41b-bef4a32fa7cd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.061050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac63be4-b2f9-45ea-b542-9a9f1f3176b2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.072694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.082449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.083247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.083558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.083989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.093695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:13.096179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.097148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.097430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.097704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] PATH_INFO: `/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.098881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] RBAC: Authorizing `identity:get_user(user_id=cee8d1e7eb98438287c0fee8c54bc3cc)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.100908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:13.101979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:13.102826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:13.103649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:13.104243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 88/178] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/users/cee8d1e7eb98438287c0fee8c54bc3cc => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.104309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 91/179] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:13.104373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.104445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.145340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.207260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2bd03-f75b-4ac1-aea0-554ebd2b47c4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.207895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663a05-4c2a-40ff-92fe-60c036a06788 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.240305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.240861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.241155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.241436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.241708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.241806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.242134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.242530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.242884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.243800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3a55ae21682648f2b1cfea192f281398, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:13.244226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.244371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.246078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 92/180] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.246269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0a32d-cf3b-4bcf-b858-2873e9d783f8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.246840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 89/181] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=3a55ae21682648f2b1cfea192f281398&role.id=011a188e9a5b4469804831444edf3f23 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4049-a11f-42c7-b65c-18c007d47cba None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4049-a11f-42c7-b65c-18c007d47cba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c4049-a11f-42c7-b65c-18c007d47cba None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 90/182] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:13.263318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac2f1-5a7f-455c-960b-d9e94c59e50d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.263918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac2f1-5a7f-455c-960b-d9e94c59e50d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.263918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afac2f1-5a7f-455c-960b-d9e94c59e50d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.263918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 91/183] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:13.263918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afac2f1-5a7f-455c-960b-d9e94c59e50d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.316309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.372065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed936b-62d0-477d-803e-e64d80a125d0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.381789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.386364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afac2f1-5a7f-455c-960b-d9e94c59e50d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.414318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.415429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.415711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.415983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.424365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.425023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.425328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.425600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.428632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:13.436576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:13.438371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 93/184] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:13.440016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.523512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.524989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01048be-dce7-4e3d-98c4-5d4dbc502741 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.549970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e44e9e-ec38-4e5f-8a9a-4de4188045bb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.571388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.585067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.585738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.586021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.586299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.587046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:13.587381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.588768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.596674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 94/185] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.597462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.629050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 92/186] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:13.630447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83a11-c534-4ace-901d-e76f806c5bea None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.630694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83a11-c534-4ace-901d-e76f806c5bea None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.630976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83a11-c534-4ace-901d-e76f806c5bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.631113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.647462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-26b83a11-c534-4ace-901d-e76f806c5bea None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:13.648636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-26b83a11-c534-4ace-901d-e76f806c5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:13.703512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecd3d1-67a3-4335-875f-754548a40c74 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.713610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 93/187] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:13.714930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.715215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.715515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.731799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:13.732961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:13.751115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.751810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.752098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.752374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.753214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=d3f391bb67354ffe82a26d3d321c1224) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:13.753530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.754988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.761370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 95/188] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/role_assignments?user.id=d3f391bb67354ffe82a26d3d321c1224&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.762152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.818570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 94/189] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:13.819638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.822481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.869259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:13.879262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf092b6-7504-4c4b-bdb2-d1a23a438f5d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.916351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.917036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.917311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.917596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:13.918372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:13.918679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:13.920619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:13.930146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 96/190] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:13.936228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:13.939038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa30ab-5a1a-4808-b4d2-50c264260f2b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:13.982720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:13.983682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:13.983960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:13.984239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.001121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.002600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.004363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 95/191] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.005139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.048103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1547c4-44c8-4bbc-839f-ba8b331b19fb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.067277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.081499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.082292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.082567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.082834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.084706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.086214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.088032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 97/192] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:13 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.088773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.112684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.112841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c13aa6-5951-45c2-89a4-f3fd77b47e5c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.147655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.148453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.148734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.149011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.150194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.151683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.153321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 96/193] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.154083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.193577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f599c6b-25bb-40c0-a521-0e77d1ebf54e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.227654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.228466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.228753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.229034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.239204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:14.247779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:14.248664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 98/194] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:14.254473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.255365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0102701-ce1d-4327-bf41-4cb961bbe800 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.279478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.288693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.289565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.289841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.290110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] PATH_INFO: `/v3/users/f9734794ff554b359699b4a12127a524` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.291316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] RBAC: Authorizing `identity:get_user(user_id=f9734794ff554b359699b4a12127a524)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.292752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.294635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 97/195] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/users/f9734794ff554b359699b4a12127a524 => generated 289 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.295413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.338075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.355885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4054ecb-8db6-4808-8815-e3f001a2e990 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.390085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.390733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.391024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.391387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.394317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041dd31-c4a7-4c18-b7da-d28fa11efcb6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.428268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.428921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.429207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.429484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.430313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=cee8d1e7eb98438287c0fee8c54bc3cc) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:14.430959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.432118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.438076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 98/196] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/role_assignments?user.id=cee8d1e7eb98438287c0fee8c54bc3cc&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.438857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.475739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.478326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.505046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.551818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60e634-d35e-49a7-8407-1e2846f1d7cd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.579033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.594346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.594943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.595229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.595526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.596241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.598125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.605467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 99/197] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:14.606327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.627030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 99/198] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:14 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:14.628216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.733625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac552b5f-56c5-45c5-8a63-3e1e3b19c53c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.765920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c873ee0-3934-420c-994b-550db6915f8e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.771558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.772249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.772529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.772800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.773559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:14.773875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.775308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.784142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.784834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 100/199] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.785678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:14.815710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.816366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.816656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.816922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.844440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:14.892663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.894192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.921433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5765b2-6ce5-4aeb-878e-e7cd1107cea7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:14.963108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:14.964379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:14.964822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:14.965103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:14.966317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:14.967866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:14.969903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 101/200] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:14.970689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.014013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 100/201] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:15.015504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.035169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.090772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896c1c0-2739-457c-a45f-cc0d60876dbc None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.105356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.127126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb27f1-f98f-4558-995c-7944e38e26e6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.129476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.130840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.130960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.131323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.132581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.134039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.135771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 102/202] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:14 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:15.137756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.171635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.172582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.173000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.173413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.174376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=c31863ac62d542dea1b63aa21f4c3691) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:15.175001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.176754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.191496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 101/203] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/role_assignments?user.id=c31863ac62d542dea1b63aa21f4c3691&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:15.193535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.250378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b163da-81a1-4cca-8d3b-7dd16e1d3ac3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.302319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.303254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.303560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.305281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.305966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.307526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.309296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 103/204] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:15.316094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.336363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f46ef-d94f-4562-a665-1529135418db None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.339629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.372618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.373202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.373489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.373767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.423502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.449679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.451187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.535803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.539085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2df85-4b09-4d7c-ba33-a1b4afa6e1ec None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.560061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 102/205] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:15.565654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.586601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.587271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.587588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.587914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.588799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, scope.project.id=d4cddc08e9474acbb1affb03326bbf68, user.id=f9734794ff554b359699b4a12127a524) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:15.589119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.590575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.598098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 104/206] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/role_assignments?user.id=f9734794ff554b359699b4a12127a524&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=f16da423656e46c386eac96321ea205e => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:15.675327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c7a51-175f-4397-ba36-92ea3a24d386 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.711786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.713128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.713406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.713676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.714957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.716477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383dd8a-dfa3-4ab0-9c4e-62f791da6356 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.722328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 103/207] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:15.725044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:15.914398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622aaff-dd5f-4ca3-be0a-7f57c9e66209 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:15.928853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:15.956586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:15.957273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:15.957553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:15.957826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:15.958724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=c31863ac62d542dea1b63aa21f4c3691) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:15.959093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:15.960916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b15f-6b44-4f7c-9f92-5d4e6740d8ad None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:15.967276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 105/208] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:15 2022] GET /identity/v3/role_assignments?user.id=c31863ac62d542dea1b63aa21f4c3691&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:16.920066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3febab1-6400-49fe-bc6c-0a5ec3c304a6 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:16.920609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3febab1-6400-49fe-bc6c-0a5ec3c304a6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:16.920609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3febab1-6400-49fe-bc6c-0a5ec3c304a6 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:16.922107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 104/209] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:16.927731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:16.928549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:16.929048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:16.947139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:16.948646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:16.988679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f704f-3f55-4832-88b0-375dc516596f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:16.988973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f704f-3f55-4832-88b0-375dc516596f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:16.989231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f704f-3f55-4832-88b0-375dc516596f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:16.991248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 105/210] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:16.996671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:16.997166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:16.997704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.018119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 106/211] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:17.019332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:17.020746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:17.042163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.084750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.119861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 106/212] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:17.151300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a608c41-4495-41ec-859e-0635fa85bc42 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.154224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.159258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.190498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.191750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.192496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.193250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.205924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:17.217138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:17.218259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:17.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:17.221586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 107/213] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:17.227918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.286353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa04a7-9e7e-4049-b113-20b32eac2b34 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.323413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.323783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.324064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.324340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.325508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:17.326985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:17.328686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 107/214] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:17.334756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.361453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d9745-2a29-47db-b70a-511780ef39ed None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.395566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.407770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.409002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.409499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.410209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.411773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:17.412576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:17.415062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:17.427021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 108/215] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:17.433461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.466229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ff013-5929-4e16-ac27-4bdd1f0c077a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.470499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.507773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.508455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.508735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.509007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.509717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:17.511157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:17.551628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 108/216] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:17 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:17.578659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ebe83-691a-4451-ad04-3c87c9c199e6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.617843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.618817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.619141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.619518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] PATH_INFO: `/v3/users/25dcd02db58846bb847c674471a495f7` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.640718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] RBAC: Authorizing `identity:get_user(user_id=25dcd02db58846bb847c674471a495f7)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:17.642732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84385be9-0110-4373-bc2e-4315df9155e2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:17.645236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 109/217] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/users/25dcd02db58846bb847c674471a495f7 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:17.651767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.713520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d33a0-6874-4ed7-bb03-bef2a70e506a None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.713802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d33a0-6874-4ed7-bb03-bef2a70e506a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.713989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d33a0-6874-4ed7-bb03-bef2a70e506a None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.715558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 110/218] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:17.721185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.721469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.721730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.740909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:17.742100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:17.777378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767d50d-bd0e-4951-85f7-ed38dec10411 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:17.779222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.829798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:17.830464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:17.830746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:17.831023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:17.835374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 111/219] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:17.851716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:17.874473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:17.928435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:17.930043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89dea2a-9175-4f33-a625-02ccf670c46e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:17.995659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65244bcb-2587-4a70-834a-0d910162c742 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.045301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.047121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.047897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.048360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.058330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 109/220] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:18.059782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1395ed4-d81a-4349-88f5-a752907ffc86 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.060076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1395ed4-d81a-4349-88f5-a752907ffc86 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.060351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1395ed4-d81a-4349-88f5-a752907ffc86 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.061891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 110/221] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:18.063662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1395ed4-d81a-4349-88f5-a752907ffc86 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.063803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:18.066808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-dbec1576-b763-4028-8804-cd235d9b2106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:18.074235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:18.075182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:18.076055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 112/222] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:18.077215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.077320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.078603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.100489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:18.102649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:18.191537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1395ed4-d81a-4349-88f5-a752907ffc86 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.195888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 113/223] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:18.200056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.220188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.241740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.242610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.242888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.243159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.244377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.246125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.248197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 111/224] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:18.249082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.319685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cbc5a-6e53-46ca-aa84-c2d1b3230859 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.343059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.357568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.358542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.358897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.359202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.360095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:18.360505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.362109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.371609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 114/225] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.373645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.384123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c46dd-71fb-446b-b9b7-1abd219e2bfd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.422532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.423410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.423690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.423961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.446095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:18.454943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:18.456042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:18.456831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 112/226] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:18.457494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.465589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.501459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b74c2f-b546-40a5-9dd9-16de1db98e66 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.537417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.538145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.538432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.538711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.539692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=25dcd02db58846bb847c674471a495f7) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:18.540021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.541600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.551357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 115/227] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/role_assignments?user.id=25dcd02db58846bb847c674471a495f7&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=011a188e9a5b4469804831444edf3f23 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.551357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.599360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def9e2-555d-4320-8278-9614a87d112a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.655251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.662243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8c5aa-d6cb-47a8-a105-6e11733b3f61 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.671897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.672805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.673092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.673719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.680325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.681834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.684039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 113/228] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.690042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.697461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:18.702381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.703361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.703806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.704217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.705119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:18.705573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.707280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.717903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 116/229] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.723823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.831659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880db53a-38a8-412a-82b1-2b17d07dcc94 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.869999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.871237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.871816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.872273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.873746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.875459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.877863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 117/230] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.883670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.884113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f770dab-f4ab-4fa2-ace7-5a34c5c02f5f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:18.932242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:18.933127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:18.933412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:18.933686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:18.936175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:18.937250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:18.939419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 114/231] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:18.944828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:18.963231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:19.018238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31a794-5f58-4e16-81be-135abc0e9773 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:19.021786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:19.061715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e5025-ae1a-471f-913a-0a45cddd0cc3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:19.064597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:19.066629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:19.067057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:19.067498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:19.069399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:19.071023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:19.073482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 118/232] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:18 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:19.081164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:19.126981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:19.127883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:19.128167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:19.128444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/c31863ac62d542dea1b63aa21f4c3691/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:19.131859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, user_id=c31863ac62d542dea1b63aa21f4c3691, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:19.134112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f1387-1ddb-4613-8de9-e2a98b778394 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:19.173573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 115/233] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:18 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/c31863ac62d542dea1b63aa21f4c3691/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:22:19.233018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13425f31-20a7-4184-b9ff-2761c29f4db5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:19.267656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:19.276483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:19.277748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:19.278230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:19.278726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/c31863ac62d542dea1b63aa21f4c3691/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:19.281286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, user_id=c31863ac62d542dea1b63aa21f4c3691, role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:19.283049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc87093-d74c-4855-9204-4fadf4f3bcb8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:19.301700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 119/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:19 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/c31863ac62d542dea1b63aa21f4c3691/roles/97a9c200717c42a4a09a99543c15fa03 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:22:20.078503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f09dd-23d1-4d41-9079-ad70d170c4fe None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.079018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f09dd-23d1-4d41-9079-ad70d170c4fe None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.079093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f09dd-23d1-4d41-9079-ad70d170c4fe None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.080745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 116/235] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:20.086208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295199e5-8d29-4d1f-b84b-fa9bc73281fc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.086616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295199e5-8d29-4d1f-b84b-fa9bc73281fc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.086807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295199e5-8d29-4d1f-b84b-fa9bc73281fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.103971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-295199e5-8d29-4d1f-b84b-fa9bc73281fc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:20.105161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-295199e5-8d29-4d1f-b84b-fa9bc73281fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:20.184990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 120/236] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:20.226941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f09dd-23d1-4d41-9079-ad70d170c4fe None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:20.265681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a37ef-0561-4a22-ac56-8a3ed485569f None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.265992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a37ef-0561-4a22-ac56-8a3ed485569f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.266235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a37ef-0561-4a22-ac56-8a3ed485569f None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.267668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 121/237] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:20.277210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.277210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.277210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.308331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:20.309685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:20.352289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f09dd-23d1-4d41-9079-ad70d170c4fe None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:20.380129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:20.393097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:20.394155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.394579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.395035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.406664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:20.414983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:20.415876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:20.416668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:20.419617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 117/238] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:20.429057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:20.463019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 122/239] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:20.487244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:20.575694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:20.609690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f1a2-c9ac-4a06-9ab2-7844831bf57a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:20.666463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5b576-cee4-42ee-9719-af9643e9e547 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:20.679852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:20.680525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.680804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.681080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.681829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:20.682130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:20.683551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:20.699550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 118/240] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:20.705223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:20.732816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:20.733627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.733909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.734181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.736100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:20.737552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:20.745978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 123/241] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:20.750022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:20.798682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:20.877102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f791929-803f-4a11-98d6-eb3732e34499 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:20.938691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353391d-1325-44c4-b63a-f984adcdb1d1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:20.952890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:20.953790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.954077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.954359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] PATH_INFO: `/v3/users/6b47905a1fa44e19be6037cfd4b2d673` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.957153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:20.974378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] RBAC: Authorizing `identity:get_user(user_id=6b47905a1fa44e19be6037cfd4b2d673)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:20.975697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:20.975987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:20.976377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:20.976661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:20.976936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:20.977704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:20.978017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 119/242] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:20 2022] GET /identity/v3/users/6b47905a1fa44e19be6037cfd4b2d673 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:20.979239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:20.983362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.015027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 124/243] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:20 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:21.100536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232eccf-fadd-4ada-b023-83570e8efcd0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:21.109331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:21.141470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:21.142468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.142949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.143463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.221279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:21.223144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:21.338757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 120/244] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:21.344271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.465640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4231bb5-cad4-4333-b6a7-d1c40323caa4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:21.502555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:21.503468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.503749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.504022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.505763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:21.507243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b0a46-25da-4e0c-b8ba-0a7a83e75cff None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:21.509349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 125/245] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:21 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:21.520426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.528858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:21.539587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93964-a0f9-4aa3-b2f5-d811f8b7fb47 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.540025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93964-a0f9-4aa3-b2f5-d811f8b7fb47 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.540420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e93964-a0f9-4aa3-b2f5-d811f8b7fb47 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.542515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 126/246] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:21.547846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.548248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.548840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.570859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:21.572363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:21.634864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:21.674638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafacd6-7a98-4115-a756-937c7ae58f68 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:21.688318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 127/247] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:21.718172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.725292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:21.725969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.726316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.726599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.727488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=6b47905a1fa44e19be6037cfd4b2d673) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:21.727801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:21.729343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4537e-3581-43f1-bd74-57111d936001 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:21.735864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 121/248] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:21 2022] GET /identity/v3/role_assignments?user.id=6b47905a1fa44e19be6037cfd4b2d673&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=f16da423656e46c386eac96321ea205e => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:21.737282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa75d3-06a8-479e-9ae7-4a0800c88dd9 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.737564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa75d3-06a8-479e-9ae7-4a0800c88dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.737838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa75d3-06a8-479e-9ae7-4a0800c88dd9 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.739338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 122/249] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:21.746369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bbe08-7448-4cf8-8ecb-3cd51c21d47a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.746690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bbe08-7448-4cf8-8ecb-3cd51c21d47a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.746931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bbe08-7448-4cf8-8ecb-3cd51c21d47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.767669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-cc2bbe08-7448-4cf8-8ecb-3cd51c21d47a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:21.768999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cc2bbe08-7448-4cf8-8ecb-3cd51c21d47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:21.849891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:21.856181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 123/250] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:21.867315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cc0-7bac-4dd5-9e5b-ce1566c2163e None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.867315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cc0-7bac-4dd5-9e5b-ce1566c2163e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.867315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a8cc0-7bac-4dd5-9e5b-ce1566c2163e None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.867315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 124/251] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:21.872565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.872857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.873124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.874132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8772517-1f9a-473e-8bdd-7270450c59be None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:21.893721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:21.897444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:21.928810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:21.929505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:21.929793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:21.930070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:21.943274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:21.953098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:21.954127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:21.954933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 128/252] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:21.958562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.986314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 125/253] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:21.987487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:21.992350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.069753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4c36d-1acb-41b8-8d7f-dd44729081e2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.100363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4975ab-3501-43cb-a140-81675aead397 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.106833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.107667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.108012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.108290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.119988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:22.128339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:22.129233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:22.130034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 129/254] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:22.131224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.136945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.145155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.146560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.146950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.147231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.148044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:22.148353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.149729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.159253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 126/255] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.164440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.273760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3b7e-3703-444c-a051-17dbdd298084 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.294565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.312284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.313071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.313354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.313625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.314487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:22.314797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.316211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.319503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51386c03-e3c3-45fb-84d5-ec07f7e0cc59 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.325698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 127/256] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.326365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.388067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.423834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.424716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.425007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.425287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.436811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099ef1a-3422-4ba3-8c14-6fdd06bd019e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.472070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:22.474216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.475433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.475936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.476428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] PATH_INFO: `/v3/users/25dcd02db58846bb847c674471a495f7` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:22.482821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:22.483944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:22.484747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:22.486462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] RBAC: Authorizing `identity:get_user(user_id=25dcd02db58846bb847c674471a495f7)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.488242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.488988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 130/257] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:22.490329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.491310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 128/258] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/users/25dcd02db58846bb847c674471a495f7 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.497104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.602411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e32fd0-11b6-4dd5-9c88-62a00effe6ec None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.657590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.658481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.658764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.659040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.661003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.662546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.664222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 131/259] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.664988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.697149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.703159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc8e8-1903-4b8b-a245-aabb24048def None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.764916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.774657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51957bc6-fba3-41c9-9fe5-f059f6bd6e3e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:22.809581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.810513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.810801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.811079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.812356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.813893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.815653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 132/260] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.816423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.824612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:22.825308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:22.825595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:22.825870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:22.826628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:22.826955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:22.828361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:22.847891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 129/261] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:22.848709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:22.961405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:22.985089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24316567-94a0-4dc5-9c81-09c91b5f8af8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.039655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf404f-9499-4c7c-ac0f-3617e9fdde61 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.046559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.047222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.047527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.047798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.088730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.090265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.090549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.090818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/25dcd02db58846bb847c674471a495f7/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.096867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, user_id=25dcd02db58846bb847c674471a495f7, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.098329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.125799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 130/262] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:22 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/25dcd02db58846bb847c674471a495f7/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:23.126676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:23.142610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.156292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.219736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:23.331681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:23.345515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9912358-82d4-4610-ac6a-a3a7a7f80c00 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.348812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 133/263] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:23.354645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:23.424556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.425823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.426329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.426803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] PATH_INFO: `/v3/users/c31863ac62d542dea1b63aa21f4c3691` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.428946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] RBAC: Authorizing `identity:get_user(user_id=c31863ac62d542dea1b63aa21f4c3691)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.443049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.446435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 131/264] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:23 2022] GET /identity/v3/users/c31863ac62d542dea1b63aa21f4c3691 => generated 283 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:23.453210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:23.533892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d96e7-6159-4d70-8c4c-d688708efdf4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.581664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0fa0e0-9b5b-4c70-a919-7025a31ba30e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.591041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.592274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.592868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.593347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.600484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:23.601784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.603844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.606424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 134/265] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:23 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:23.623138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:23.625022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.625691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.625979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.626260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.697412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.698060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:23.698913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.771650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3db90-ac35-45fc-b21c-179096806d19 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.806958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 132/266] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:22:23.808264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:23.822684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:23.867405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.868145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.868439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.868723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.869561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=c31863ac62d542dea1b63aa21f4c3691) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:23.869877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.871353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.897025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 135/267] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:23 2022] GET /identity/v3/role_assignments?user.id=c31863ac62d542dea1b63aa21f4c3691&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:23.939021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbedb-0f47-4785-a508-3dea42d5b3ef None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:23.993182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:23.994075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:23.994361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:23.994640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:23.995943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:23.997426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da6c71-358b-4783-abb3-f4f658a47752 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:23.999529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 133/268] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:23 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:24.005486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.143509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21ebc-d019-4872-b89c-cae96da5b723 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:24.171142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:24.190202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:24.191268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.191773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.192235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.193829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=c31863ac62d542dea1b63aa21f4c3691) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:24.194354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:24.196050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285ecb5-507f-4ede-b450-557093ebc32e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:24.207862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 136/269] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/role_assignments?user.id=c31863ac62d542dea1b63aa21f4c3691&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:24.295949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d489b5-2f89-4757-ab14-416e622380b7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.296456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d489b5-2f89-4757-ab14-416e622380b7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.296528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d489b5-2f89-4757-ab14-416e622380b7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.298233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 134/270] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:24.312955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.313592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.314064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.329213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec7dc3-d661-40d6-a2d7-98a717d81b68 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.329853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec7dc3-d661-40d6-a2d7-98a717d81b68 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.330323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec7dc3-d661-40d6-a2d7-98a717d81b68 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.332275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 135/271] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:24.340852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.341484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.341967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.359037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:24.364894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:24.371181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:24.373205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:24.478534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 137/272] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:24.501795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 136/273] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:24.507155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.525982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.541057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:24.624007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91812c-eb00-48a6-bf95-f12e207270a2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:24.639757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:24.641731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e969-791e-431b-b0dc-082c850c6af0 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:24.664601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:24.665923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.666355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.666816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.681558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:24.683715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:24.684624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.685004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.685391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:24.691808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:24.692885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:24.693791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:24.696530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 138/274] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:24.702653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:24.703194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:24.711606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:24.712586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:24.713426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 137/275] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:24.718895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.809436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:24.829334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e61650-faff-4d06-b98d-22beedb6fef0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:24.852984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f812c-031a-4d1c-a83f-1d7dd114e267 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:24.863723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:24.864394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.864825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.865126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.865872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:24.866176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:24.867591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:24.875971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 139/276] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:24.881583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.900749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:24.901437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:24.901821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:24.902160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:24.903036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:24.903443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:24.904860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:24.914692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 138/277] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:24.920223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:24.925003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:24.993210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cab047-329b-4003-bf23-c7b85dd5197e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.029595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.030462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.030737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.030917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7efa91-5cc2-4308-ad5a-b13303158dbd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.031017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] PATH_INFO: `/v3/users/1318de6e07c94489aeb8c2b0adcada58` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.045724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] RBAC: Authorizing `identity:get_user(user_id=1318de6e07c94489aeb8c2b0adcada58)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.047178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e814ad-23e2-4337-8e64-470751b8d35b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.048894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 140/278] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/users/1318de6e07c94489aeb8c2b0adcada58 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:25.050100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b476-fba0-421c-bf32-3722c4fc3a9a None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.050379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b476-fba0-421c-bf32-3722c4fc3a9a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.050646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330b476-fba0-421c-bf32-3722c4fc3a9a None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.057075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 141/279] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:25.057075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330b476-fba0-421c-bf32-3722c4fc3a9a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.073230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:25.079713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.080786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.081110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.081412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] PATH_INFO: `/v3/users/6b47905a1fa44e19be6037cfd4b2d673` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.082926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] RBAC: Authorizing `identity:get_user(user_id=6b47905a1fa44e19be6037cfd4b2d673)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.085300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb46ab0-d550-4ce6-ac2b-5ac1bb651880 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.087013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 139/280] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:24 2022] GET /identity/v3/users/6b47905a1fa44e19be6037cfd4b2d673 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:25.088359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847487-aed3-4890-9daa-709055a9973a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.088646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847487-aed3-4890-9daa-709055a9973a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.088920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f847487-aed3-4890-9daa-709055a9973a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.106013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6f847487-aed3-4890-9daa-709055a9973a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:25.107305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6f847487-aed3-4890-9daa-709055a9973a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:25.157365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:25.167279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330b476-fba0-421c-bf32-3722c4fc3a9a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.178890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 140/281] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:25.179591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f847487-aed3-4890-9daa-709055a9973a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.215743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.216404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.216694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.216960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.217818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, user.id=1318de6e07c94489aeb8c2b0adcada58, scope.system=all) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:25.218127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.219594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.225047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 142/282] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/role_assignments?user.id=1318de6e07c94489aeb8c2b0adcada58&scope.system=all&role.id=011a188e9a5b4469804831444edf3f23 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:25.225967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.322703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f847487-aed3-4890-9daa-709055a9973a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.357850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:25.358772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49820e-cd79-48b3-ab0c-23e5ba3053b1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.366952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.367986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.368335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.368681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.370786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.372856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.375158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 141/283] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:25.384295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.410049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.411603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.411887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.412146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.413796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.415704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.418204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 143/284] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:25.423916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.468283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:25.494104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b897026-b4b9-49e3-a122-95f2a93ebc85 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.531324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.532209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.532491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.532763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/6b47905a1fa44e19be6037cfd4b2d673/roles/f16da423656e46c386eac96321ea205e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.534896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, user_id=6b47905a1fa44e19be6037cfd4b2d673, role_id=f16da423656e46c386eac96321ea205e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.536705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856b8db-ef62-42f1-93af-39a463f22ef4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.555394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 142/285] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:22:25 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/users/6b47905a1fa44e19be6037cfd4b2d673/roles/f16da423656e46c386eac96321ea205e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:25.561556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077b6d4-a933-407f-b674-5e0f29678805 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.601544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.602611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.603087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.603581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.604504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.606405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70dcd7-a55e-4f15-8bf9-22123de519ae None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.634226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 144/286] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:25 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:22:25.863345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c41c3-cc91-4384-b59c-0287762cc828 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.863695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c41c3-cc91-4384-b59c-0287762cc828 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.863868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c41c3-cc91-4384-b59c-0287762cc828 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.865520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 143/287] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:25.870832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.871117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.871436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.888997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:25.890168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:25.962400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 145/288] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:25.986138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c41c3-cc91-4384-b59c-0287762cc828 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:26.052236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:26.096253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c41c3-cc91-4384-b59c-0287762cc828 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:26.131680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:26.132833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:26.133304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:26.133766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:26.144429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:26.153094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:26.153973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:26.154763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:26.157434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 144/289] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:26.164538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:26.186879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:26.293308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbc52b-b031-4ad9-a387-7f4cd8e8dcb4 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:26.328848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:26.329517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:26.329800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:26.330073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:26.330819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:26.331132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:26.332685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:26.342089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 146/290] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:26.347908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:26.503137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156604-250c-4905-b03d-acbcddaa5537 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:26.521198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:26.539570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:26.540796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:26.541268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:26.541731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] PATH_INFO: `/v3/users/25dcd02db58846bb847c674471a495f7` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:26.543139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] RBAC: Authorizing `identity:get_user(user_id=25dcd02db58846bb847c674471a495f7)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:26.545093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:26.547067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 145/291] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:26 2022] GET /identity/v3/users/25dcd02db58846bb847c674471a495f7 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:26.556485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:26.646369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:26.678398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37f8b-ed6f-421e-b30d-1da03f5a5af0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:26.721852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:26.727323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:26.728141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:26.728614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:26.822441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:26.824012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:26.920827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:26.960231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 147/292] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:26.966041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:27.104799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5763d8-3e70-4892-875f-9acc45a027b5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:27.151944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:27.153047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.153467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.153905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.155224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:27.156792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508c927-7906-47ac-bf13-cc8b11e2b358 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:27.158922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 146/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:26 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:27.164741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:27.324786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742198-5221-4ee7-86c9-feadbec21922 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:27.367087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:27.367694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.367974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.368247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.369078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=25dcd02db58846bb847c674471a495f7) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:27.369389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:27.370837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9ac89-98bd-41c4-b8f5-0d0a40c6326b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:27.377194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 148/294] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:27 2022] GET /identity/v3/role_assignments?user.id=25dcd02db58846bb847c674471a495f7&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=011a188e9a5b4469804831444edf3f23 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:27.619087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815d84-bf47-4af8-8347-35a7112f246b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.621868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815d84-bf47-4af8-8347-35a7112f246b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.621868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815d84-bf47-4af8-8347-35a7112f246b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.621868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 147/295] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:27.628122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57ced8-c0a1-423f-8402-f41fb629f518 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.628409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57ced8-c0a1-423f-8402-f41fb629f518 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.628689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57ced8-c0a1-423f-8402-f41fb629f518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.653650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-da57ced8-c0a1-423f-8402-f41fb629f518 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:27.655623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-da57ced8-c0a1-423f-8402-f41fb629f518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:27.689056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:27.727244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 149/296] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:27.751380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815d84-bf47-4af8-8347-35a7112f246b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:27.779742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bb6-5b13-49b7-9c0b-169ad417dafb None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.780018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bb6-5b13-49b7-9c0b-169ad417dafb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.780454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3bb6-5b13-49b7-9c0b-169ad417dafb None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.782189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 150/297] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:27.789276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.789879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.790433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.808812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:27.810265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:27.817126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:27.869468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815d84-bf47-4af8-8347-35a7112f246b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:27.884990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 151/298] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:27.907812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:27.908831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:27.909258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:27.909673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:27.911642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:27.922814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:27.932356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:27.933458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:27.934462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:27.936591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 148/299] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:27.943166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.032965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.038490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4d609-77eb-4cb9-8f1f-e38a0f1f5a0e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.082270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a9aec-c5f0-4992-9e32-bfff6bfed589 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.083161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.084507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.085882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.086340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.097752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.101660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:28.113186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:28.114096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:28.114980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:28.117801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 152/300] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:28.123859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.126284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.127104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.127622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.128002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.129404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:28.129404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.131647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.141497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 149/301] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:28.147452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.236718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6dd5b6-d3f3-47a2-af25-e98d9df20767 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.278609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595a56b-1d5d-45fc-8f09-cfc01cbb9960 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.279620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.280375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.280743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.281225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.282913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:28.283323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.284814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.300692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 153/302] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:28.302101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.321545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.322719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.323026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.323300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] PATH_INFO: `/v3/users/1318de6e07c94489aeb8c2b0adcada58` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.324529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] RBAC: Authorizing `identity:get_user(user_id=1318de6e07c94489aeb8c2b0adcada58)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.325983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.328625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 150/303] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/users/1318de6e07c94489aeb8c2b0adcada58 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:28.333819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.335363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.390121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.425165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29eef-8f60-4bd4-bfb2-9c302c91de66 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.447664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba72de4-560b-4066-86cb-979b72f36125 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.466543 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.467460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.467900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.468309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] PATH_INFO: `/v3/users/6b47905a1fa44e19be6037cfd4b2d673` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.469744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] RBAC: Authorizing `identity:get_user(user_id=6b47905a1fa44e19be6037cfd4b2d673)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.471337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.478080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 154/304] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/users/6b47905a1fa44e19be6037cfd4b2d673 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:28.479330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.483423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.484929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.484929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.484929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] PATH_INFO: `/v3/system/users/1318de6e07c94489aeb8c2b0adcada58/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.486236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1318de6e07c94489aeb8c2b0adcada58, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.487655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7261d45-f123-4384-9670-3d86d505a380 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.498472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 151/305] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Jul 20 09:22:28 2022] PUT /identity/v3/system/users/1318de6e07c94489aeb8c2b0adcada58/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:28.542800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da88-a0ee-4709-bc7d-600d549d928d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.543248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da88-a0ee-4709-bc7d-600d549d928d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.543750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962da88-a0ee-4709-bc7d-600d549d928d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.546329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 152/306] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:28.553595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f85bb-5cd0-4dbf-9e43-33cbeb8ebb33 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.554098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f85bb-5cd0-4dbf-9e43-33cbeb8ebb33 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.554746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f85bb-5cd0-4dbf-9e43-33cbeb8ebb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.575190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f55f85bb-5cd0-4dbf-9e43-33cbeb8ebb33 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:28.576434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f55f85bb-5cd0-4dbf-9e43-33cbeb8ebb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:28.586705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.600381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849e68-c41d-4654-b164-32915fb2fa68 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.639621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.640344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.640559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.640833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.659711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 153/307] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:28.661236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6b9a-5d22-45ec-a023-6391647eb01e None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.662282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6b9a-5d22-45ec-a023-6391647eb01e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.662570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd6b9a-5d22-45ec-a023-6391647eb01e None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.664074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 154/308] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:28.669616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.670345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.670644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.688193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:28.689379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:28.719433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.719830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.721310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.763677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 155/309] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:28.764650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.821661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.837128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 155/310] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:28.838176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.896510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f0d22-c2eb-4ce1-9b12-7c1a71457af9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.938035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.938832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.939114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.939413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:28.940586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:28.941549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47ecef-5b9b-4246-8f70-27c0d3f7f39d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:28.942016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:28.943813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 156/311] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:28.944576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:28.982687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:28.984586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:28.985422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:28.985706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:28.985976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.000272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:29.008208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:29.009135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:29.009931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 156/312] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:29.012104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:29.056582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c7550-bbc1-4907-9429-6a51d6e65172 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.065124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:29.093182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.094043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.094326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.094596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.095820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.097271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.099331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 157/313] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:28 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:29.100103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:29.126291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296c3b2-5175-456d-bc50-e058ab0e1786 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.163093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.163725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.164003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.164277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] PATH_INFO: `/v3/groups` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.164961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.166503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.183534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 157/314] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Jul 20 09:22:29 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:29.190658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:29.211677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f45a6-4902-4d86-97b6-af2f7350cbc2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.215744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:29.255917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.257472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.258051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.258589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.259986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:29.260162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.261819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.272614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 158/315] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:29.277862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:29.339620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2882c-0a76-409e-b473-fe7cbf5ad813 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.352669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:29.391828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.392494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.392775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.393059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.394054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327, user.id=6b47905a1fa44e19be6037cfd4b2d673) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:29.394671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.396816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.398250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643eb58-ae92-4694-a91e-d18d5eff888e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.403023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 158/316] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:29 2022] GET /identity/v3/role_assignments?user.id=6b47905a1fa44e19be6037cfd4b2d673&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=f16da423656e46c386eac96321ea205e => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:29.449696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.450981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.451595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.452073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] PATH_INFO: `/v3/users/afecb04092b74f4aa8c03b793c39661d` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.468916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] RBAC: Authorizing `identity:get_user(user_id=afecb04092b74f4aa8c03b793c39661d)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.470753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27ad76-145a-4fd5-83b5-1694d4da0459 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.473137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 159/317] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:29 2022] GET /identity/v3/users/afecb04092b74f4aa8c03b793c39661d => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:29.479852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:29.597213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489375d2-52b6-4d67-9344-c084b3fc7efb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:29.635193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:29.635917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:29.636199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:29.636474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:29.637330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, user.id=afecb04092b74f4aa8c03b793c39661d, scope.system=all) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:29.637649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:29.639162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce3dc8-318e-4502-ac8c-cc7c0e77627a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:29.646960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 159/318] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jul 20 09:22:29 2022] GET /identity/v3/role_assignments?user.id=afecb04092b74f4aa8c03b793c39661d&scope.system=all&role.id=f16da423656e46c386eac96321ea205e => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:30.565902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04c85-1597-4db0-8563-50d3244db18b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:30.566407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04c85-1597-4db0-8563-50d3244db18b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:30.566481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead04c85-1597-4db0-8563-50d3244db18b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:30.568397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 160/319] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:30.573424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:30.573823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:30.573977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:30.591153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:30.592349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:30.622393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:30.662885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 160/320] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:30.686444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04c85-1597-4db0-8563-50d3244db18b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:30.712306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:30.792567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead04c85-1597-4db0-8563-50d3244db18b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:30.825790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:30.826582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:30.826858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:30.827126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:30.837645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:30.845421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:30.846338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:30.847154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 161/321] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:30.853967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:30.954925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e54ae-e006-4acc-9f2d-11f0a942f292 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:30.988356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:30.988950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:30.989260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:30.989536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:30.990408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] RBAC: Adding query filter params (name=member) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:30.990711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:30.992110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:31.000270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 161/322] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:31.006399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:31.089918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:31.107630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99d35a-7799-4c30-a056-be34022051a2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:31.140560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:31.141429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.141709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.141980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] PATH_INFO: `/v3/users/1318de6e07c94489aeb8c2b0adcada58` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.143157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] RBAC: Authorizing `identity:get_user(user_id=1318de6e07c94489aeb8c2b0adcada58)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:31.144609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0019ad-4afc-4968-b3a8-895fc2c941fc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:31.146521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 162/323] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:31 2022] GET /identity/v3/users/1318de6e07c94489aeb8c2b0adcada58 => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:31.152026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:31.182824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:31.259721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59323af1-d50c-4403-91a3-0093e403e184 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:31.294048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:31.295006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.295348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.295663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.296529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] RBAC: Adding query filter params (role.id=011a188e9a5b4469804831444edf3f23, user.id=1318de6e07c94489aeb8c2b0adcada58, scope.system=all) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:31.296880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:31.298352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268993a-d5f4-46da-b600-d5c9c04fe41c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:31.318171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 162/324] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jul 20 09:22:31 2022] GET /identity/v3/role_assignments?user.id=1318de6e07c94489aeb8c2b0adcada58&scope.system=all&role.id=011a188e9a5b4469804831444edf3f23 => generated 510 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:31.334725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5ea7-8656-45c2-a52d-a73856a1d221 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.335005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5ea7-8656-45c2-a52d-a73856a1d221 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.335327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f5ea7-8656-45c2-a52d-a73856a1d221 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.336967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 163/325] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:31.342638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.342948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.343202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.360931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:31.362147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:31.443903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 163/326] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:31.465991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f5ea7-8656-45c2-a52d-a73856a1d221 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:31.576199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f5ea7-8656-45c2-a52d-a73856a1d221 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:31.608633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:31.609425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.609708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.609984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.611143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:31.612614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca51cbe-f7b7-4082-9b1f-f80ffc940fd5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:31.614947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 164/327] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:31.620666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:31.638472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:31.674127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65987cf-05a7-455d-96e4-e3defb99beb4 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.674408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65987cf-05a7-455d-96e4-e3defb99beb4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.674685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65987cf-05a7-455d-96e4-e3defb99beb4 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.676088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 165/328] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:31.682102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.682386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.682666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.699144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:31.700439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:31.714518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:31.725719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848726c-b7a6-4029-b210-75fcd87aaf7b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:31.759536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:31.760200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.760483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.760758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] PATH_INFO: `/v3/groups` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.761453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:31.762900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:31.765950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 166/329] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:31.787318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 164/330] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Jul 20 09:22:31 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:31.791026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:31.895562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720c17c-6f62-48b2-a26c-d93ecb090215 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:31.931704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:31.932508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:31.932786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:31.933057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:31.943564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:31.951346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:31.952423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:31.953397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 167/331] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:31.958428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:32.041897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:32.069409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc509ce3-e32f-4c69-bbbb-823946f60ba5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:32.104241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:32.105240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:32.105521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:32.105792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:32.106535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:32.106906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:32.108389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:32.118008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 165/332] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:32.123739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:32.138208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:32.237029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dce09-5760-430c-86ff-7c2e29b5f5fe None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:32.272848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:32.273731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:32.274013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:32.274351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] PATH_INFO: `/v3/users/afecb04092b74f4aa8c03b793c39661d` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:32.275720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] RBAC: Authorizing `identity:get_user(user_id=afecb04092b74f4aa8c03b793c39661d)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:32.277178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324e71c-e090-42dd-9646-6e2fb29c9899 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:32.279139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 168/333] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:32 2022] GET /identity/v3/users/afecb04092b74f4aa8c03b793c39661d => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:32.284848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:32.391524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6184bd-3496-453f-b828-51044775c681 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:32.427164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:32.428589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:32.429048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:32.429495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] PATH_INFO: `/v3/system/users/afecb04092b74f4aa8c03b793c39661d/roles/f16da423656e46c386eac96321ea205e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:32.431176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=afecb04092b74f4aa8c03b793c39661d, role_id=f16da423656e46c386eac96321ea205e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:32.432771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaf495-c49c-4ca6-9dd5-c28da24fc5bc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:32.450533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 166/334] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Jul 20 09:22:32 2022] PUT /identity/v3/system/users/afecb04092b74f4aa8c03b793c39661d/roles/f16da423656e46c386eac96321ea205e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:34.297213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3c45-c6a3-442c-a2c5-c0e4568ecf6e None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.298311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3c45-c6a3-442c-a2c5-c0e4568ecf6e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.298804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf3c45-c6a3-442c-a2c5-c0e4568ecf6e None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.300693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 169/335] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:34.306565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f60-cdae-43a4-8ec1-0e23fa8814d5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.307445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f60-cdae-43a4-8ec1-0e23fa8814d5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.307932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37034f60-cdae-43a4-8ec1-0e23fa8814d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.313418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:34.332054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-37034f60-cdae-43a4-8ec1-0e23fa8814d5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:34.333996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-37034f60-cdae-43a4-8ec1-0e23fa8814d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:34.405734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 167/336] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:34.429508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf3c45-c6a3-442c-a2c5-c0e4568ecf6e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:34.469018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74532b35-d716-4aa9-b7d0-c3d2eda77fd6 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.469564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74532b35-d716-4aa9-b7d0-c3d2eda77fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.469997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74532b35-d716-4aa9-b7d0-c3d2eda77fd6 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.471581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 168/337] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:34.478559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.479057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.479536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.499639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:34.502010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:34.596151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:34.604155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 169/338] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:34.633914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:34.659999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf3c45-c6a3-442c-a2c5-c0e4568ecf6e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:34.719763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:34.750611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c13991-0564-4864-9457-b0308f4e3055 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:34.789676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:34.790903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.792521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.792958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.803986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:34.804626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:34.804800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.805085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.805359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-64ec1d04-1ad9-43ec-b1a9-c1e51a37445b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:34.813780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:34.814908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:34.815947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:34.823918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 170/339] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:34.825429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bd8a8-e698-4215-84cc-d3d00454b29d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.825874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bd8a8-e698-4215-84cc-d3d00454b29d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.826299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bd8a8-e698-4215-84cc-d3d00454b29d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.828200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 171/340] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:34.829110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bd8a8-e698-4215-84cc-d3d00454b29d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:34.834288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-0e86023e-b8f4-4c35-b8e5-2bc8d0ce7f90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:34.849870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:34.850929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:34.851947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:34.858779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 170/341] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:34.860337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.860795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.861227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.881484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:34.882903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:34.940764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bd8a8-e698-4215-84cc-d3d00454b29d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:34.958218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 171/342] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:34.959677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:34.967696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:34.979145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:34.980249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:34.980775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:34.981294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:34.982550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:34.983092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:34.984844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.000261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 172/343] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:35.001439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.047830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.081521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848b27c-51fd-4b74-b92a-918b4d399c66 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.109366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35de6b-d6ff-48c5-8cbe-4c7481137180 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.121456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.121963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.122249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.122524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.123275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:35.123606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:35.125067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.141150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 172/344] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:35.142069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.150915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.152207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.152712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.153180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.165487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:35.175375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:35.176334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:35.177734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:35.180628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 173/345] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:35.182011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.247686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.266426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385afd6e-5781-4ed4-8d2c-280df1f64483 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.307263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.314928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.315634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.315917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.316189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.317283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d53ef-47e7-41f3-a263-3fd893e9c4bd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.364994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.365997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.366806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.367363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.406634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:35.408276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.495546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:35.497293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.534227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 173/346] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:35.535876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.575094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.645149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.651110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f13f77-ea8e-4806-8d04-0482046fa9e4 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.691174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.692233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.692891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.693330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.694271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:35.694765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:35.697021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.706323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 174/347] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:35.707585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.759363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 174/348] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:35.759573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.828255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f3712-ac11-4ccb-abb5-3edbecc69841 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.839638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.870950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.874244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.874244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.874244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.875701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:35.877700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:35.880303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 175/349] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:35.881244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:35.893216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc4f50-f962-4f2c-abce-532ace0d3448 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:35.937767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:35.938344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:35.938620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:35.938895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:35.956667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.009623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd866a-7097-4b6d-9d44-5fb1e8e2f614 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.026393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.028664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.049785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.051036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.055729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.055729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.055729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.055729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65ec9-584b-4e2d-96cb-dfce5b0d0691 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.059407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 176/350] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.060788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77d4e-d4ef-4542-9455-5a36ed801ec8 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.061082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77d4e-d4ef-4542-9455-5a36ed801ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.061347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b77d4e-d4ef-4542-9455-5a36ed801ec8 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.062880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 177/351] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:36.063978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e319f2f-1ada-4a67-9ee0-b2eabcdf5fcd None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.064309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e319f2f-1ada-4a67-9ee0-b2eabcdf5fcd None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.064586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e319f2f-1ada-4a67-9ee0-b2eabcdf5fcd None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.065795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 178/352] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:36.066792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e319f2f-1ada-4a67-9ee0-b2eabcdf5fcd None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.107380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.164201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 175/353] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:36.165967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.219881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e319f2f-1ada-4a67-9ee0-b2eabcdf5fcd None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.234136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.263886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.264757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.265040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.265315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.272554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.274002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb158e4-912b-446d-be9d-143decd95a41 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.275781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 179/354] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.277195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.277485 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.277767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.296373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:36.297603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:36.312337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7cb1c3-d00e-46aa-8dbe-b7c64bdf891a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.364541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.365530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.365813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.366087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.368123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.369914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e43a84-402d-4a5b-a657-394fb15c7128 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.374600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 176/355] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.374600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831897ba-3770-46c5-9250-5247a80626cb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.374962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831897ba-3770-46c5-9250-5247a80626cb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.375254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831897ba-3770-46c5-9250-5247a80626cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.391092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 180/356] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:36.391998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.398236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-831897ba-3770-46c5-9250-5247a80626cb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:36.399524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-831897ba-3770-46c5-9250-5247a80626cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:36.421411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.486985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 177/357] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:36.488138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831897ba-3770-46c5-9250-5247a80626cb None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.532743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde510d-f460-468b-8910-1b1fddc36920 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.535643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.580458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.581608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.581887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.582157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.583957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.585600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.587812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 181/358] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.588457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.601691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831897ba-3770-46c5-9250-5247a80626cb None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.639611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.640269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.640548 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.640816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.641726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=d4cddc08e9474acbb1affb03326bbf68) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:36.642041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.644477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.651128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 178/359] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.651957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.702412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.716107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c58a7-6037-4856-a02b-e3b36be1bee9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.759823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.760903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.761191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.761465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.762655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=d4cddc08e9474acbb1affb03326bbf68) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:36.762964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:36.764470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:36.773356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 182/360] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=011a188e9a5b4469804831444edf3f23 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:36.774205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.792081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5089253-2461-4640-8355-21fb92374bf0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.804253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:36.848305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.849722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.850002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.850271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.864745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:36.873245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:36.874234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:36.875153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 179/361] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:36.875853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:36.926462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246611a-e631-49be-92bd-6299922ef5cb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:36.977410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:36.979012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:36.979968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:36.980414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:36.994048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ada44490-dd67-4327-8b76-c3b195271707 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:37.002894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:37.003927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:37.005595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 183/362] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:37.006466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf4bb-53e9-464c-9615-ed929224b63d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.006466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf4bb-53e9-464c-9615-ed929224b63d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.007039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf4bb-53e9-464c-9615-ed929224b63d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.008865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 184/363] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:37.009584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbf4bb-53e9-464c-9615-ed929224b63d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.011971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55976397-62f5-409f-8dcf-c4726b9ab55b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.035644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.055619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.056487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.056766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.057037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.058329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.059804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f863ad73-c7c6-46df-ac08-6568f41a42b7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.061625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 180/364] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:36 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:37.062589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.062859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.063132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.084184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:37.085363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:37.116670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.146545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbf4bb-53e9-464c-9615-ed929224b63d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.168595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 181/365] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:37.169803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.201901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.202593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.202875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.203149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.203964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:37.204461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.205761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.222958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 185/366] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:37.222958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.302563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f652cef-b25b-4500-80a9-a5f48e8eeab9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.330912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.340390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.341179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.341465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.341737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.342497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:37.342805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.344376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.356283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 182/367] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:37.357274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.363859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20a0d-5081-4463-9397-5eaf45b1c796 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.411662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.412989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.414210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.414693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.415221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.416446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:37.417009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.419850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.426294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 186/368] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=011a188e9a5b4469804831444edf3f23 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:37.431576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.493915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff6d0b-ee25-4d85-b77c-57d4194d67b2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.543404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.544908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.545488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.545984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.560952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:37.569604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:37.570558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:37.571395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:37.574301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 183/369] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:37.575534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.668774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4f460-a374-438c-b1d8-8b3c34ff8e9e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.704089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.717917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.718569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.718848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.719116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.727889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24783c-05db-4bca-882d-08ff09c8f539 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:37.743334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:37.795813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:37.796744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:37.797023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:37.797292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] PATH_INFO: `/v3/users/afecb04092b74f4aa8c03b793c39661d` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:37.799609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] RBAC: Authorizing `identity:get_user(user_id=afecb04092b74f4aa8c03b793c39661d)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.801046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.803940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 184/370] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/users/afecb04092b74f4aa8c03b793c39661d => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:37.804504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:37.847488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:37.849541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:37.993630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca04eb-86d0-4a22-b7f3-7334c63fc632 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.038825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 187/371] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:38.039492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.058589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.059259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.059563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.059836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.060587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:38.060891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.065299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.081710 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 185/372] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:38.082332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.111388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:38.121310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:38.226337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b4be-edea-46b3-ad37-1e9826ab5e2d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.264371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f02f-5b82-4256-b999-8b8aea877330 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.294623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.295282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.295581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.295916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.296748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] RBAC: Adding query filter params (role.id=f16da423656e46c386eac96321ea205e, user.id=afecb04092b74f4aa8c03b793c39661d, scope.system=all) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:38.297105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.298548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.322860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 188/373] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/role_assignments?user.id=afecb04092b74f4aa8c03b793c39661d&scope.system=all&role.id=f16da423656e46c386eac96321ea205e => generated 510 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:38.323721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.340169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.341063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.341347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.341623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.342981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.344464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.346525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 186/374] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:38.356368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.521514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de6a07-636c-4a71-a98a-932da83226f5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.524645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:38.577966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:38.602254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.602909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.603187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.603490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.613211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b30ad5-4b53-45ff-9cec-95af2beb567b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.696292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.697082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.697366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.697740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.701819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.703322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.705016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 187/375] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:38.711182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.718783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.720518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.828042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:38.835579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 189/376] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:38.841510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:38.914314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ac16a-7527-473d-b594-2c156ac29bf6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:38.958534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:38.959106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:38.959422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:38.959698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:38.961060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:38.961379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:38.963133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:38.970204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 188/377] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:38.983678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c2dc-d501-477c-b303-2c7a7f85e040 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:39.031051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:39.032229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.032745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.033166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] PATH_INFO: `/v3/projects/c93172815ff14813957f03bf548f6155` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.040349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] RBAC: Authorizing `identity:get_project(project_id=c93172815ff14813957f03bf548f6155)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:39.042096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:39.044432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 190/378] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:38 2022] GET /identity/v3/projects/c93172815ff14813957f03bf548f6155 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:39.050793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:39.079553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:39.188102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0df05-93aa-42b5-a35c-ebe0db60b2d0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:39.231323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:39.232139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.232423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.232700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] PATH_INFO: `/v3/groups/58a4274a131b4d6e90ae54ce0130897f` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.241183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] RBAC: Authorizing `identity:get_group(group_id=58a4274a131b4d6e90ae54ce0130897f)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:39.242663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf8968-f2b9-4f51-9f87-7d4d1bd24bf7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:39.244752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 189/379] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/groups/58a4274a131b4d6e90ae54ce0130897f => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:39.255765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:39.333931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:39.350488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe897-2889-4d54-9e97-7449ac0b1ae8 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.350828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe897-2889-4d54-9e97-7449ac0b1ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.351053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe897-2889-4d54-9e97-7449ac0b1ae8 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.352603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 190/380] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:39.353568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992049-ed4a-4869-a893-6ec136fb2aee None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.353813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992049-ed4a-4869-a893-6ec136fb2aee None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.354092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992049-ed4a-4869-a893-6ec136fb2aee None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.355333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 191/381] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:39.357679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.357965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.358241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.382683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:39.384207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:39.389208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30c2a-d0b7-4df9-8cb2-987e87809684 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:39.439533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:39.440259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.440544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.440819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.441662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] RBAC: Adding query filter params (group.id=58a4274a131b4d6e90ae54ce0130897f, role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=c93172815ff14813957f03bf548f6155) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:39.441977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:39.444279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c145b7-386e-4cc7-a2a9-bfa4be32c758 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:39.453868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 191/382] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/role_assignments?group.id=58a4274a131b4d6e90ae54ce0130897f&scope.project.id=c93172815ff14813957f03bf548f6155&role.id=743f7cb41c634460b2981782f093ad19 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:39.455666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.456153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.456597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.480771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:39.482360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:39.512707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 192/383] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:39.553980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:39.567272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:39.596277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 192/384] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:39.626147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:39.720600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a69539-2d58-4c1d-b200-b0270d1f3bb9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:39.737605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:39.794992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc6edd-8ec4-48ca-98f7-ea176d4cc66b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:39.813331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:39.814167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.814445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.814717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.837827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:39.838917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:39.839390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:39.839853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:39.846329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:39.853895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:39.855158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:39.856057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:39.856865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:39.860872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 193/385] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:39.867558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:39.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:39.869653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:39.870374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:39.872490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 193/386] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:39.879358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.024975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57986dd0-9934-4004-b9ca-899e95c5a924 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.033260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.047690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.072928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ede7c7-161c-4d1b-9041-70122db52326 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.077354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.078035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.078318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.078594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.079380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:40.079692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.081070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.091905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 194/387] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:40.099607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.137353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162288bd-a537-423f-b139-021f205b0728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.138042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162288bd-a537-423f-b139-021f205b0728 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.138329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162288bd-a537-423f-b139-021f205b0728 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.138605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162288bd-a537-423f-b139-021f205b0728 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.139390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162288bd-a537-423f-b139-021f205b0728 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:40.139704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162288bd-a537-423f-b139-021f205b0728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.141088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162288bd-a537-423f-b139-021f205b0728 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.157831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 194/388] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:40.180868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162288bd-a537-423f-b139-021f205b0728 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.230089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ebb94-85ec-49df-bd71-c9712453f6a0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.283504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.284056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.284337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.284611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.287535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.288981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24a5c0-0f53-4491-a2d9-fc287eaa00cc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.291058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 195/389] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:40.292194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54d967-d373-4d0e-b4bb-4b58aef25d98 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.292479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54d967-d373-4d0e-b4bb-4b58aef25d98 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.292754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54d967-d373-4d0e-b4bb-4b58aef25d98 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.298259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 196/390] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:40.299417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.299700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.299976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.305331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162288bd-a537-423f-b139-021f205b0728 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.323911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:40.325163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:40.349725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.351426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.352249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.352738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.354810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.357006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.361537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.363398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 195/391] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:40.365446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.388606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.414904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 197/392] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:40.416067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.500711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f0be8-dd1e-4157-8993-d66d6bc01877 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.549636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.550814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.551278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.552282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6da91-277b-4653-acbe-638b391b2860 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.552800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.557236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.558912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.560884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 196/393] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:40.566834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.599833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.600973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.601263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.601540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.602930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.605086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.606976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 198/394] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:40.612033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.675125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.682799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.696853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d245b92-0c29-421c-b0d5-3c09a72ca764 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.735271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.736117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.736399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.736679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.748622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127067a-3e97-41a7-9d5d-0629ac893f07 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.749986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:40.760742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:40.761621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:40.762404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 197/395] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:40.763052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.796442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:40.797320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:40.797602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:40.797877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:40.800817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, group_id=09ed9d9afd8d45c7aaccb2275775efdf, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:40.802441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:40.827341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 199/396] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jul 20 09:22:40 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:40.827341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:40.931857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b3532-e1da-40bc-83ea-632193c35393 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.953159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19612f-2928-4bb7-9f16-539c9548fc6d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:40.965415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:40.999157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.000267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.001156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:41.002432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.002932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.012375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:41.012684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:41.014093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:41.028044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.029057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.029349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.029602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.034101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4cddc08e9474acbb1affb03326bbf68, group_id=09ed9d9afd8d45c7aaccb2275775efdf, role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:41.035590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:41.036587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 200/397] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:41.040269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:41.063319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 198/398] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jul 20 09:22:40 2022] PUT /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/97a9c200717c42a4a09a99543c15fa03 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:41.170338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451bd20-cadc-44ec-b0a4-0a9241f416c2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:41.215912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.216864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.217143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.217415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.218663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:41.220127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:41.222089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 201/399] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:41 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:41.227520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:41.340500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba564-bc66-4b95-8137-5f3e1935cbc7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:41.377684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.378955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.379469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.380702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.382293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:41.383991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c988be4-224b-4514-8576-ec1ff60ed630 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:41.386330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 199/400] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:41 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:41.396409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:41.485275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:41.538959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb3fe5-e1a4-4699-81c4-fbb59a119cfa None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.539613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb3fe5-e1a4-4699-81c4-fbb59a119cfa None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.540133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb3fe5-e1a4-4699-81c4-fbb59a119cfa None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.543395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 200/401] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:41.547269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.547816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.548302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.566584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd46fdd-e38a-4aaa-9282-df04584b857e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:41.577725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:41.579071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:41.605208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.606668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.607112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.607580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.613300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, group_id=09ed9d9afd8d45c7aaccb2275775efdf, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:41.615656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:41.641337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 202/402] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jul 20 09:22:41 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:41.677699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:41.706262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 201/403] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:41.741448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:41.897554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:41.902388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58ed9-1ab7-4a57-b182-d67c767314c5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:41.949021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:41.949841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:41.950121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:41.950396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:41.967099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-54012652-3af8-4f22-970a-3b6a05cf0c62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:41.975365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:41.976299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:41.977123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 203/404] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:41.982699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.068366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e5b7e-2808-4bf0-83fb-c5c6b4cd759d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.068930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e5b7e-2808-4bf0-83fb-c5c6b4cd759d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.069403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e5b7e-2808-4bf0-83fb-c5c6b4cd759d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.071164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 204/405] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:42.078227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.078514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.078792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.097314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:42.098591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:42.152464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd3b93-0a9a-4f46-92ca-bda720f966fd None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.175432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 205/406] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:42.203801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.204279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.204897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.205318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.205739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.206759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:42.207216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.209207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.230448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 202/407] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:42.236046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.320653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:42.363817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:42.367164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b1a4-6d19-441c-8e18-11698159d1c6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.394648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf1879-da86-497d-95fe-fc370b262bc7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.445701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.445701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.445701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.445701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.447454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.448268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.448529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.448801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.450195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.452754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.458458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 203/408] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:42.460387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.475395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:42.484975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:42.485884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:42.486670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 206/409] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:42.493400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.645863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b71e-739d-4ff5-8e06-c08c0912875c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.652503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537df588-b058-4c70-a9eb-37fcc1be8b74 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.670642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:42.697491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.698075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.698359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.698635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.699400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:42.699710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.701091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.704545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.705410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.705691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.705964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.707142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.708789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.713095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 204/410] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:42.718504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.720570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 207/411] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:42.732826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.758179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:42.840980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca49fee-1965-44b3-bf3d-cd4ca4e2cecd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.873864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52230a-94a3-4a5d-b8e6-ff21d880373d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:42.917462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.918344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.918619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.918884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.920303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:42.921674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:42.921947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:42.922214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] PATH_INFO: `/v3/groups/58a4274a131b4d6e90ae54ce0130897f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:42.923787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] RBAC: Authorizing `identity:get_group(group_id=58a4274a131b4d6e90ae54ce0130897f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.925206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.927370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 208/412] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/groups/58a4274a131b4d6e90ae54ce0130897f => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:42.928088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=3e5a178e4ffa4f368ce2589db6fbc327, group_id=09ed9d9afd8d45c7aaccb2275775efdf, role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:42.929505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:42.933438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:42.948685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 205/413] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jul 20 09:22:42 2022] PUT /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327/groups/09ed9d9afd8d45c7aaccb2275775efdf/roles/97a9c200717c42a4a09a99543c15fa03 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:43.063505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:43.092168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9f2a-761c-4a8c-b509-4ad4ff8f7fbc None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:43.148955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:43.149836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.150118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.150390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] PATH_INFO: `/v3/projects/c93172815ff14813957f03bf548f6155` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.151658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] RBAC: Authorizing `identity:get_project(project_id=c93172815ff14813957f03bf548f6155)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:43.153095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558209-422c-4c09-9dce-33d9de81b79d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:43.155148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 209/414] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:42 2022] GET /identity/v3/projects/c93172815ff14813957f03bf548f6155 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:43.161601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:43.230689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:43.292035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bba428-dcfc-4729-8c46-83f884753b3c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:43.327464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:43.328538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.328837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.329159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] PATH_INFO: `/v3/projects/c93172815ff14813957f03bf548f6155/groups/58a4274a131b4d6e90ae54ce0130897f/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.331259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] RBAC: Authorizing `identity:create_grant(project_id=c93172815ff14813957f03bf548f6155, group_id=58a4274a131b4d6e90ae54ce0130897f, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:43.332915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cb904-4f33-4965-aa6a-6ce05891832d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:43.355470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 206/415] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Jul 20 09:22:43 2022] PUT /identity/v3/projects/c93172815ff14813957f03bf548f6155/groups/58a4274a131b4d6e90ae54ce0130897f/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:43.518727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4eab7-dbeb-478a-befe-d526d0e6b5f7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.519445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4eab7-dbeb-478a-befe-d526d0e6b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.519923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4eab7-dbeb-478a-befe-d526d0e6b5f7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.522618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 210/416] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:43.527932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b7c-8f9a-44a6-aa2c-d0e312c1603a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.528506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b7c-8f9a-44a6-aa2c-d0e312c1603a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.528983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b7c-8f9a-44a6-aa2c-d0e312c1603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.553259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8aae6b7c-8f9a-44a6-aa2c-d0e312c1603a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:43.554891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8aae6b7c-8f9a-44a6-aa2c-d0e312c1603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:43.632362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 207/417] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:43.656631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4eab7-dbeb-478a-befe-d526d0e6b5f7 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:43.748625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9930c0-075d-4030-8377-4cf807f836be None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.748988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9930c0-075d-4030-8377-4cf807f836be None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.749518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9930c0-075d-4030-8377-4cf807f836be None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.753408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 208/418] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:43.758099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.758639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.758980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.785447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4eab7-dbeb-478a-befe-d526d0e6b5f7 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:43.792332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:43.793672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:43.839392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:43.840731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:43.841753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:43.842932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:43.843545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:43.867465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:43.880028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:43.881111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:43.882024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:43.886310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 211/419] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:43.893212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:43.904637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:43.918854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 209/420] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:22:43.947430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.017072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc576c3-7f64-41a3-88db-0eedd1730e51 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.058453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.059057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.059368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.059645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.060396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:44.060700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:44.062063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:44.080427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 212/421] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:44.086987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.127491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543f681-9e02-423d-83ff-5cc20ba99812 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.180979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.181808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.182086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.182374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.203682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.204492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-d1123691-d038-489a-829f-f9b72e420503 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:44.215861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:44.216777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:44.217592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 210/422] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:44.223546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2ab7-27f4-4507-9392-fae4ad86637e None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.223973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2ab7-27f4-4507-9392-fae4ad86637e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.224317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea2ab7-27f4-4507-9392-fae4ad86637e None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.225831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 211/423] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:44.230043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea2ab7-27f4-4507-9392-fae4ad86637e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.239968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13158df8-77aa-4465-b227-c42f9c4d20f2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.265214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.275622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.276278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.276561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.276836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.354743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea2ab7-27f4-4507-9392-fae4ad86637e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.372516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:44.374278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:44.398073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.398750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.399171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.399602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.400362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:44.400670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:44.402725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdcf81-4529-4c74-9409-0bc193313466 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:44.419667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 212/424] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:44.421065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.421354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.421628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.454560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:44.455963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:44.458867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.507604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 213/425] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:44.508471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.539414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 213/426] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:44.540611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.573173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.640601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd402f-925e-409f-864c-1ce78652e30d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.669157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e3c3b-8442-40ca-abcb-6b26feff9097 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.678122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.679312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.679773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.680243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.715135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.716132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.716483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.717076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.718886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.719824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:44.721491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:44.725638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 214/427] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:44.727235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.757301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:44.759095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:44.831055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:44.851047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6cb-ed5f-4ed7-8cd7-1ed4f728dc0e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:44.889508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:44.890819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:44.891384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:44.891853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:44.893987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 214/428] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:44.894905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.904144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:44.912864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:44.913731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 09:22:44.914528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:44.917301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 215/429] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:44.918363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:44.977288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.018647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e546ae7-07d1-4647-a4ee-a0213e30ee28 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.030694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d0b8f-a340-4f64-8af7-4e520988f49c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.062514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.063704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.063985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.064257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.065698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.067369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.069257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.070336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.070898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.071419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] PATH_INFO: `/v3/projects/d4cddc08e9474acbb1affb03326bbf68` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.072799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] RBAC: Authorizing `identity:get_project(project_id=d4cddc08e9474acbb1affb03326bbf68)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.074041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 215/430] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.074661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.074878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.077121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 216/431] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:44 2022] GET /identity/v3/projects/d4cddc08e9474acbb1affb03326bbf68 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.078251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.095850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.189862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc7687-b395-4425-989c-6df903235f5d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.201395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddbe6b-3ddc-48d2-827d-d66e43b58839 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.227423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.228417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.228903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.229370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.230527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=d4cddc08e9474acbb1affb03326bbf68) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:45.231092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.232892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.239784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 217/432] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=011a188e9a5b4469804831444edf3f23 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.240802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.250757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.251482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.251764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.252036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.252778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] RBAC: Adding query filter params (name=member) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:45.253079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.255002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.258028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.275066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 216/433] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.285548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.378556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.406590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b12da2-c8d6-4a08-a6ab-358c988417c9 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.453446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.454317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.454847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.455321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.456842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.459177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ee2e-0918-4d58-b054-9eded686d665 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.461393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 218/434] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.468035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b00f-43fe-4f4c-a810-5bb5ea1b666b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.468550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b00f-43fe-4f4c-a810-5bb5ea1b666b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.469215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702b00f-43fe-4f4c-a810-5bb5ea1b666b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.470858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 219/435] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:45.471792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702b00f-43fe-4f4c-a810-5bb5ea1b666b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.495463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d02489-7852-423b-a90c-f2f419d0eddc None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.540379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.541801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.542269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.542838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.603324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702b00f-43fe-4f4c-a810-5bb5ea1b666b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.636773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.640523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.643017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.653776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.654713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.655140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.655622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.656602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=d4cddc08e9474acbb1affb03326bbf68) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:45.657060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.658746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d1c2c-dc6a-430a-a2a0-1e010575586a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.666153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 220/436] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=d4cddc08e9474acbb1affb03326bbf68&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.667819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.668309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.668832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.688832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:45.690372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:45.695736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.773424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 221/437] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:45.791987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 217/438] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:45.797717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.807450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.933537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994f2ac-fe4f-437f-9adc-ca029a239b38 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.938529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:45.943728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560a225-6c0d-407c-8559-b02b71d2bdec None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:45.980424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.981256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.981496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.981772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.982973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:45.984059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e09060-940e-413c-961f-25293f19b586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:45.984448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd48afc-8847-447b-834a-09f2fbc98865 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:45.984875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09060-940e-413c-961f-25293f19b586 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.985162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09060-940e-413c-961f-25293f19b586 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.985439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e09060-940e-413c-961f-25293f19b586 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.986161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 222/439] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:45.987583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788946-0360-4bfc-8aa4-b33dd822aab2 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:45.987876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788946-0360-4bfc-8aa4-b33dd822aab2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:45.988154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788946-0360-4bfc-8aa4-b33dd822aab2 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:45.990235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 223/440] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:45.993128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3788946-0360-4bfc-8aa4-b33dd822aab2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:45.996608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e09060-940e-413c-961f-25293f19b586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:46.002130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-28e09060-940e-413c-961f-25293f19b586 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:46.006192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:46.007343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 218/441] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:46.008370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.009243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.009243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.033524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:46.034729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:46.115264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 219/442] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:46.116132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.120602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3788946-0360-4bfc-8aa4-b33dd822aab2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.154830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.155718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.155995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.156261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.157415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.158869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.160516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 224/443] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:45 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:46.160516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.209211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.221872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5ea46-574b-4b8f-ad1a-429d7fe9760b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.237193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.256102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.256771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.257049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.257326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.258072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:46.258377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.259907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.262838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0eb26-692e-4e4a-9a5d-77f605f7ba80 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.267966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 220/444] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:46.268746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.296791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.297578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.297859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.298136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.307980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:46.315600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:46.316484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 225/445] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:46.318072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.372404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b8d46-9960-4186-8708-82aaf084fe75 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.405302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.405949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.406229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.406511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.407360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=011a188e9a5b4469804831444edf3f23, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:46.407675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.409097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.415275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 221/446] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=011a188e9a5b4469804831444edf3f23 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:46.416070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.416314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb80c6a-97c4-4316-bc5f-574e25c19b6c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.447143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.466870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.467574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.467855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.468124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.471330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.541975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bd78e-3b61-401b-b15d-8ad1393ef795 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.547780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.549368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.582028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.584174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.584174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.584174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.584526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:46.584834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.586207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.598377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 222/447] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:46.603961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.665379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 226/448] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:46.671664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.702618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.711577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.729983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a3cc-75e0-4e53-82d7-7b421580ae4b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.778374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.779040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.779361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.779642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.791826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d62da-78ea-4128-9746-6ce61ac5a280 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.825531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.826312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.826589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.826866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] PATH_INFO: `/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.828073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] RBAC: Authorizing `identity:get_project(project_id=3e5a178e4ffa4f368ce2589db6fbc327)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.829503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.831199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 227/449] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/projects/3e5a178e4ffa4f368ce2589db6fbc327 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:46.836784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.847963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.850097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.924614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.936591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0111b-8b09-4b85-810a-21f83d126e9c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:46.938078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:46.949279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 223/450] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:22:46.954751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:46.971366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:46.972227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:46.972508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:46.972779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] PATH_INFO: `/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:46.974634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] RBAC: Authorizing `identity:get_group(group_id=09ed9d9afd8d45c7aaccb2275775efdf)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:46.976091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:46.977699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 228/451] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/groups/09ed9d9afd8d45c7aaccb2275775efdf => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:46.983337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:47.050598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13011859-dfdb-4533-bc0d-b19eb8379132 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:47.080152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9ad1f-a33e-49bf-8f3c-f1538b02c7e8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:47.083676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:47.084526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:47.084803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:47.085070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] PATH_INFO: `/v3/projects/c93172815ff14813957f03bf548f6155` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:47.086243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] RBAC: Authorizing `identity:get_project(project_id=c93172815ff14813957f03bf548f6155)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:47.087713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:47.089375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 224/452] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/projects/c93172815ff14813957f03bf548f6155 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:47.094660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:47.113462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:47.114627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:47.114995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:47.115313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:47.116124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] RBAC: Adding query filter params (group.id=09ed9d9afd8d45c7aaccb2275775efdf, role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:47.116442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:47.117866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:47.123684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 229/453] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:46 2022] GET /identity/v3/role_assignments?group.id=09ed9d9afd8d45c7aaccb2275775efdf&scope.project.id=3e5a178e4ffa4f368ce2589db6fbc327&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:47.140699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:47.198786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557632-1357-411b-a815-9327ebfc8263 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:47.234679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:47.235595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:47.235875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:47.236146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] PATH_INFO: `/v3/groups/58a4274a131b4d6e90ae54ce0130897f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:47.237385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] RBAC: Authorizing `identity:get_group(group_id=58a4274a131b4d6e90ae54ce0130897f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:47.238848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8edfb-05fe-4e0c-9eb7-65b9786f85cc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:47.240581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 225/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Jul 20 09:22:47 2022] GET /identity/v3/groups/58a4274a131b4d6e90ae54ce0130897f => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:22:47.246554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:47.283478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:47.353401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15dc95-492b-4e7d-aed2-9238a93a3294 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:47.387605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:47.388249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:47.388525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:47.388791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:47.389615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] RBAC: Adding query filter params (group.id=58a4274a131b4d6e90ae54ce0130897f, role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=c93172815ff14813957f03bf548f6155) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:47.389918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:47.391384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8591614-f485-4c71-9d02-befd1c61e2cd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:47.397672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 230/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Jul 20 09:22:47 2022] GET /identity/v3/role_assignments?group.id=58a4274a131b4d6e90ae54ce0130897f&scope.project.id=c93172815ff14813957f03bf548f6155&role.id=743f7cb41c634460b2981782f093ad19 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:50.029644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccaa27-d61c-403a-84e7-ecef8a7d30ea None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.030658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccaa27-d61c-403a-84e7-ecef8a7d30ea None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.031155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccaa27-d61c-403a-84e7-ecef8a7d30ea None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.033013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 226/456] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:50.039111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa38c12-95ab-4ce2-a616-924ecb6ea0ee None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.040030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa38c12-95ab-4ce2-a616-924ecb6ea0ee None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.040504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa38c12-95ab-4ce2-a616-924ecb6ea0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.063010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-faa38c12-95ab-4ce2-a616-924ecb6ea0ee None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:50.064576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-faa38c12-95ab-4ce2-a616-924ecb6ea0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:50.133502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 231/457] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:50.158346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccaa27-d61c-403a-84e7-ecef8a7d30ea None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.237567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2c544-22d4-45dc-bb19-00ae289f82cd None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.237894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2c544-22d4-45dc-bb19-00ae289f82cd None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.238106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2c544-22d4-45dc-bb19-00ae289f82cd None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.240688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 232/458] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:22:50.248041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df50-e382-4d6c-966e-93334f71e169 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.248628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df50-e382-4d6c-966e-93334f71e169 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.248915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4df50-e382-4d6c-966e-93334f71e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.267115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2fb4df50-e382-4d6c-966e-93334f71e169 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:50.268920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2fb4df50-e382-4d6c-966e-93334f71e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:50.269895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccaa27-d61c-403a-84e7-ecef8a7d30ea None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.284250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.295728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.308529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.309317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.309781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.310192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.321584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.323205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-527121d8-291c-4367-8c46-df723ecdf5d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:50.342348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:50.343257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:50.344133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:50.344133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:50.344133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.344133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:50.344356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 233/459] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:50.344356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb151b-425c-4199-aac3-7639da73d67b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.345023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb151b-425c-4199-aac3-7639da73d67b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.345371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb151b-425c-4199-aac3-7639da73d67b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.346677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 227/460] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:50.351163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 234/461] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:50.352760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb151b-425c-4199-aac3-7639da73d67b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.353220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461719ad-e21f-4879-8710-5c056df40897 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.353718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461719ad-e21f-4879-8710-5c056df40897 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.354209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461719ad-e21f-4879-8710-5c056df40897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.372932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-461719ad-e21f-4879-8710-5c056df40897 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:50.374413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-461719ad-e21f-4879-8710-5c056df40897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:50.444945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 228/462] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:50.446208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461719ad-e21f-4879-8710-5c056df40897 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.476001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeb151b-425c-4199-aac3-7639da73d67b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.518760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.519890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.519890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.520144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.520871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:50.521181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.522579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.534831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 235/463] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:50.539318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.550318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.558989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461719ad-e21f-4879-8710-5c056df40897 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.559631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.595826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.597074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.597660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.598731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.610186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.612390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:50.632324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:50.633244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:50.634200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:50.634200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:50.634200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.634200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:50.636153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 229/464] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:50.637115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.641730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1879948-e860-444c-b91a-59d84502ce62 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.676122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.677043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.677467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.677839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.689029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.690714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:50.708816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:50.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:50.710554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:50.710554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:50.710554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:50.710554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:50.714887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 236/465] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:50.715806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.749963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47931b7-4d35-4ad8-9ca0-e8c0a17683c1 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.788925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.789113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.789566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.789849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.790125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.790832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.792295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.814094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 230/466] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:50 2022] POST /identity/v3/users => generated 248 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:50.814911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.824274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:50.826840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82903d9-48e5-4342-8465-2b47921836ff None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:50.872715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:50.873430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.873762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.874043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.874968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:50.875388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:50.877004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc2ad-acca-4c6c-b6e3-cb84ddb86386 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:50.892694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 237/467] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:50.894360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3532f5-3656-4997-82bc-f1a9297303e6 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:50.894648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3532f5-3656-4997-82bc-f1a9297303e6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:50.895457 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3532f5-3656-4997-82bc-f1a9297303e6 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:50.897268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 238/468] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:50.901442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3532f5-3656-4997-82bc-f1a9297303e6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:50.964239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c482e-316b-4e33-b3ef-e0cd969295db None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.015343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.016376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.016847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.017247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.018090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:51.018544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.020500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc76bf-90df-4d89-8de6-2477f0e564e7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.026371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3532f5-3656-4997-82bc-f1a9297303e6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.033120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 231/469] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:51.034228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182dde3-a5f8-4c50-928c-9fad775d9aa3 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.034509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182dde3-a5f8-4c50-928c-9fad775d9aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.034785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182dde3-a5f8-4c50-928c-9fad775d9aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.052149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3182dde3-a5f8-4c50-928c-9fad775d9aa3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:51.053380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3182dde3-a5f8-4c50-928c-9fad775d9aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:51.069930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.078032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.079023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.079524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.079969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.080833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.083036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.106319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 239/470] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:50 2022] POST /identity/v3/users => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:22:51.107339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.122380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.144525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 232/471] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:51.148423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f82-c7af-4020-aaf2-ae1a92fa4ae6 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.148715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f82-c7af-4020-aaf2-ae1a92fa4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.148995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c3f82-c7af-4020-aaf2-ae1a92fa4ae6 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.150689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 233/472] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:51.159756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.160088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.160315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.192438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:51.194216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:51.247578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0216c4-007d-4efe-8b5b-77bc24df3f04 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.288129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 234/473] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:51.289127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.297296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.297597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.297878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.298149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.298857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.300372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.325940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 240/474] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:51 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:51.332639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.395697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.419882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47972104-07c4-475a-bfc6-32d114d5d3c5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.460365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.466744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551de7b-14fb-4b67-a8c8-d7303379e539 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.477570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.479178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.480011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.480531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.495943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.498456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.505698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.506464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.506811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.507154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.508011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.509824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:51.523138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:51.524599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:51.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:51.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:51.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:51.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:51.525699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.sql.core [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 09:22:51.525699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 09:22:51.525699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [parameters: {'id': '2f310c6d91294c54967bb5197e1818e6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 09:22:51.525699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80185) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 09:22:51.527748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 235/475] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:51.534252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result.read() Jul 20 09:22:51.541900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:22:51.542853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application yield resource Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise self.value Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application session.commit() Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 09:22:51.543891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.session.flush() Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application compat.raise_( Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 09:22:51.545072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:22:51.545990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result.read() Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:22:51.546836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [parameters: {'id': '2f310c6d91294c54967bb5197e1818e6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:51.548300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 09:22:51.549127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.550097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 241/476] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jul 20 09:22:51 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:51.550097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.667003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f64b6-fe32-4c76-9087-2816fedcd7de None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.686043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07597bad-c5d2-4cf8-a075-418d19254de2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.703340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.703740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.704219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.704595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.704969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.705780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:51.706182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.708222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.721591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 236/477] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:51.724730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.725944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.726465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.727198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.729241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.741375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:51.751246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:51.752193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 09:22:51.752992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:51.756027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 242/478] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:22:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:51.763664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:51.771725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:51.853000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7626f7-4830-4702-9673-922155b3b1ef None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.891126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.892491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.893143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.893930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.895821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.898275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334e3d2-8b68-490e-a6bd-d5caf9d6a288 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.900027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791746e0-ca44-421f-bbcf-22b23e581746 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:51.924656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 237/479] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:51 2022] POST /identity/v3/users => generated 249 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:51.937042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:51.937845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:51.938195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:51.938538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:51.943096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:51.943096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:51.943096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab9e9e-e158-49b2-ab83-60e4c5370ee2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:51.951382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 243/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:53.337823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a17cb-4d7c-4d59-9a5b-6c4ea8116ebb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.338344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a17cb-4d7c-4d59-9a5b-6c4ea8116ebb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.338344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a17cb-4d7c-4d59-9a5b-6c4ea8116ebb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.340144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 238/481] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:53.345432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.345819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.345984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.362841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:53.364573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:53.392015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:53.456766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 244/482] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:53.492723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a17cb-4d7c-4d59-9a5b-6c4ea8116ebb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:53.634045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:53.648915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a17cb-4d7c-4d59-9a5b-6c4ea8116ebb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:53.694886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:53.696076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.696580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.697050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.709711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a02b24ee-1002-4eb4-b28f-b4a1facf6b00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:53.718332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:53.719344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:53.720137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:53.727386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 239/483] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:53.729528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:53.876909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4cd35-3be5-4037-9f68-22863ed1057d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:53.914535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:53.915136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.915450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.915729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.916486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:53.916749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9cc8-62a3-4687-93bf-7d91ca2b33dc None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.916854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:53.917318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9cc8-62a3-4687-93bf-7d91ca2b33dc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.917801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9cc8-62a3-4687-93bf-7d91ca2b33dc None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.918185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08e5d5-5a47-4277-a4b1-53f34b8bd277 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:53.931318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 245/485] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:22:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:53.931502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 240/485] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:53.937197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.937645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.938093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.952299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d9cc8-62a3-4687-93bf-7d91ca2b33dc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:53.956057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:53.957978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:53.977997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.069085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 246/486] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 09:22:54.136793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d9cc8-62a3-4687-93bf-7d91ca2b33dc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.162188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.212482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.214336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.216309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.216741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.241881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.243591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-e6117a00-74d8-4843-ac16-cd8555f97d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:54.268790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:54.269726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:54.271193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:54.271193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:54.271193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:54.271193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:54.271193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 241/487] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:54.276269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90753-77d5-4457-b15c-63aae7b14fa6 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.276568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90753-77d5-4457-b15c-63aae7b14fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.276855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90753-77d5-4457-b15c-63aae7b14fa6 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.278324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 242/488] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:54.279327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c06e6-7559-4c6a-9379-fb1b3c564e24 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.279666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c06e6-7559-4c6a-9379-fb1b3c564e24 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.279946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c06e6-7559-4c6a-9379-fb1b3c564e24 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.281399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 243/489] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:54.282115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c06e6-7559-4c6a-9379-fb1b3c564e24 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.285010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.317781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ba4e1-902b-4e8b-b84d-8d083e6491ac None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.354667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.355485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.355764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.356033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.367390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-df2bf742-da77-4b43-af63-43325e71f9ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:54.376263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:54.377152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:54.377954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 247/490] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:54.379013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.379013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.379327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.396491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:54.397670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:54.400487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.401310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c06e6-7559-4c6a-9379-fb1b3c564e24 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.439146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.440067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.440519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.440851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.441526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:54.441906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.443311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f6489-5b67-4d08-8755-29775c4a9403 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:54.460369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 244/491] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:54.460369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.460369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.460369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.475896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:54.477028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:54.481774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 248/492] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:54.484232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.541383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.552909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 245/493] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:54.553946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.603831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbea17-764d-44e2-85a4-81b868f978db None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.639812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.640473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.640747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.641015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.641755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:54.642054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.643438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:54.653201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 249/494] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:54.654374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.660107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.662505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d19bf-c660-45f5-b75f-7637bccec1c4 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.697457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.698261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.698538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.698810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.706073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.708049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:22:54.724867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:54.725827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 246/495] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:54.726874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.767574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d37120-04c5-473b-a564-29467fea1111 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.781630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.802871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.803771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.804049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.804325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.816523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.818777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:54.828891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24871d2d-0111-4c2f-a172-a57adbeba5e9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:54.835926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:54.836958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:54.837881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:54.837881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:54.837881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:54.837881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:54.841955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 250/496] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:54.842692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.862419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.863207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.863515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.864305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.874088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-655fdf4c-7ff1-4414-bfc6-56c97c57111f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:54.882686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:54.883789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:54.884823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 247/497] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:54.885694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436df5-d040-41eb-b102-e786f2aaf614 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.885694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436df5-d040-41eb-b102-e786f2aaf614 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.885694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436df5-d040-41eb-b102-e786f2aaf614 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.886926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 248/498] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:54.887645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436df5-d040-41eb-b102-e786f2aaf614 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:54.904019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:54.949585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8613c-a40b-47ff-b7cc-1c13c0835810 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.982748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:54.983491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:54.983769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:54.984042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:54.991967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436df5-d040-41eb-b102-e786f2aaf614 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:54.995521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:54.996974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.004083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:55.015256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:55.016314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 251/499] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:55.017482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.028213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.028882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.029164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.029439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.030283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:55.030592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.032080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.069692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 249/500] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:22:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:55.070511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.118324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46617354-dfc9-4899-91c6-992b0b9aaec4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.125426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.155655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.156308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.156592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.156864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.157572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:55.157875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.159229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256549-bbbf-47da-92e7-4118d3f0ff98 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.168383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0abd3e-1f0f-415d-9dd7-d1d21ebf1fd0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.170593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 252/501] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:55.171824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.172106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.172379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.187945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:55.189124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:55.208270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.209052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.209248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.209522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.210267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:55.210590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.212048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.219813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 250/502] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:55.220588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.251082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.258181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 253/503] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:55.259098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.320881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c5f40-3358-45f0-87bc-e632af2985b4 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.354633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.355319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.355618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.355874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.356592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:55.356896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.358263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.366198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.371159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 251/504] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:55.371971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.372605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0beb3-c259-4019-b780-7a0f0d13f4cb None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.406354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.407000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.407281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.407581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.461842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.470844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0690f-0858-4649-bf27-831ab301c91d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.480393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.481861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.505746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.506418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.506701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.506975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.507721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.509148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.529143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 252/505] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:22:55 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:22:55.529948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.595413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.604164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 254/506] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:55.608312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.645407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1ec88-9c3a-41ac-bcb5-8fd6269c670e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.687488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.688338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.688559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.688823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.702368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.704399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.715124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.727685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8233-a17d-4464-a3d8-38102fb9eb14 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:55.727973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:55.728951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:55.730041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:55.730041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:55.730041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.730041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:55.730041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 253/507] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:55.730373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.767593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.768621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.769056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.769477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.781836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:55.790459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:55.791476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:55.792465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:55.794748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 255/508] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:55.795691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.840431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1468fdf-5f97-4d30-8ba7-917ea9062eb5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.856035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.878493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.879692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.879974 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.880245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.888510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.889951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.903529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f63924-84dd-4d4f-a975-27c1c30ed271 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:22:55.905094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:55.906030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:55.906993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 254/509] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:55.907594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.950054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.950945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.951134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.951470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.963332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.964791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.972220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:55.983686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:55.984560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:55.985428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:55.985428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:55.985428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:55.985428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:55.985428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 256/510] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:55.985716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.023470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa29c9f-9425-49dd-8345-061841b9c740 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.059441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.060104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.060388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.060658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.061374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.061675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.063039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.075949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 255/511] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.076769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.104110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19a7df-4d04-4c70-8ab2-a4882ac56f05 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.113341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.141031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.141816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.142096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.142382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.143122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.143468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.145869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.158762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 257/512] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.158762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.186160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf8344-8be5-455b-a3a6-ed2de38e03df None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.206603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.222532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.223382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.223667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.223943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.224779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.225085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.226506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.245163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 256/513] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.245871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.275616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe784-920d-4ec7-85ec-e1cfb62b3280 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.312014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.312837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.313125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.313406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.314129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.314440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.315867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.329243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 258/514] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.330070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.359246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e77dd-4bbd-4310-ad16-89455650e7f4 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.364155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.395877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.396830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.397117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.397408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.404574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.406011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:22:56.420724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:56.421641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:56.422565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 257/515] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:56.423120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.451097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4239f4-56dd-49c5-993a-622f4ec5ff28 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.470899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.490332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.491105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.491419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.491703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.502670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.504142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:56.522849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:56.523835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:56.524698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:56.524698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:56.524698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:56.524698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:56.524910 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 259/516] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:56.525860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.541695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7f82-301e-4757-bf2e-c9eb31d1c43e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.578568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.579218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.579543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.579817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.609396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.628226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80402ec3-d5b7-4bd9-8db4-ae91a24afad2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.654945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.656444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.666778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.667609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.667893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.668172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.675854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.678394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:22:56.693825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:56.694772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:56.695670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 260/517] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:56.696155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.706436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.773365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 258/518] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:56.774214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.807069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03fc2e-a970-426a-8206-427b0f8429df None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.843464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.844128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.844411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.844689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.845512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.845826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.847251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.879194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7537-e38e-487a-8272-5023561ffeae None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:56.880738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.889234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 261/519] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.889234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.917831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:56.918488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:56.918772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:56.919048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:56.920169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:56.920584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:56.921960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:56.929349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:56.935513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 259/520] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:56.936313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:56.994430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36549-e511-45fe-a300-6a659d636bf4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.029963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.030622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.030942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.031179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.031992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:57.032304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.033745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.040490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807463a2-3045-4d2e-87c8-1e3d9a461a8a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.058378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 262/521] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:57.058378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.081420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.082349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.082910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.083192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.094611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.096108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:57.114727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:57.115799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 260/522] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:57.116874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.124440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.170014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989b810-7c4c-4f47-bb86-8b4a39da222c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.174607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.209972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.210620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.210899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.211172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.228661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea74c1c-2e19-47bc-a63e-3451d162bb01 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.266885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.267727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.268009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.268281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.276724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.278162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.286346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.288007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:22:57.292546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:57.293656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:57.294973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 261/523] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:57.295581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.355005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.401367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 263/524] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:57.402244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.411163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa814576-906a-4ffe-a48b-f132752d473c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.423263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.450893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.451586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.451869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.452144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.452972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=f2e6dffba870453fab836e0ccece7708) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:57.453287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.454718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.471629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 262/525] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/role_assignments?user.id=f2e6dffba870453fab836e0ccece7708&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:57.471629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.536799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5d431-1861-493b-8caf-60fed275115c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.578018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.578975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.579362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.579717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.580526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:57.580907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.583013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.596202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 264/526] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:57.597199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.615337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0d8ff-1692-41eb-ac49-ee4df5e8c4f0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.622143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.661983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.662660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.662945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.663218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.664092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:57.664400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.665839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.684834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 263/527] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:57.685666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.716910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.720766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a5e5d-94b8-4bca-9fd6-5c498029792c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.761388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.762498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.762498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.762755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.773818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.775275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.791648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc3831-b5ac-47df-b374-31ac904c8877 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-e9bf5624-eb8f-4dcc-a0e1-a4cca065f37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:57.793623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:57.794732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 265/528] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422366c4-5c30-4ff0-953b-ce66557b7344 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422366c4-5c30-4ff0-953b-ce66557b7344 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.797011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422366c4-5c30-4ff0-953b-ce66557b7344 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.802956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 266/529] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:57.802956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422366c4-5c30-4ff0-953b-ce66557b7344 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.836014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.836798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.837082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.837357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.844520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:57.846053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:22:57.860877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:57.861776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:57.862621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 264/530] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:57.863077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:57.882865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.923540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422366c4-5c30-4ff0-953b-ce66557b7344 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:57.964626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:57.965010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:57.965663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:57.965964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:57.966220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:57.976046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3f5c4-f2ed-48dd-bf71-4a681ef319c2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.011907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.012569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.012854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.013127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.013851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.014147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.015546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27819940-ac66-40c0-a5b9-567480afdc6d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.027482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 265/531] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.028768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.029050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.029325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.042502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.043992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.046850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:58.048077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:58.127309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 266/532] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:58.127309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.127570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.155338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 267/533] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:22:58.156108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.219855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.240207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ebd97f-8a75-4bb9-a12e-0d767b470603 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.258782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bde32-4df5-4829-868b-cc2c83c9c456 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.279491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.280149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.280431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.280705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.281479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.281794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.283245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.294786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.295781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.296202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.296636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.302076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 267/534] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.302880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.308532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.310126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:22:58.326077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:58.326946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:58.327883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 268/535] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:58.328355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.361629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.419383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b44199-5116-4613-9cc3-3ade356d1e94 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.441225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7ec5a-d263-422a-8e2f-15b1d9a13f3f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.461521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.462448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.462829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.463199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.474684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:22:58.476824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.477650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.477933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.478209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:22:58.483573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:58.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:22:58.485674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:58.487156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 268/536] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:58.487807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.491252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.496670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.498132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:58.517328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:58.518373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:58.519564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:58.519564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:58.519564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:58.519564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:58.525635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 269/537] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:58.525635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.602636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb545-26d0-436a-ae20-b88b0ed1eb0a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.615583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.632596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afca98f-181e-4604-9f7e-70dd48b73bed None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.641515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.642426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.642705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.642983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.643851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.644159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.646095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.652259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 269/538] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.653036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.678889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.679943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.680507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.681116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.681989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.682346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.683894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.702524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 270/539] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.703534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.736509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.777319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac0216-f339-440e-8935-69fdf3eb43cb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.821344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.822395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.823007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.823528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.824506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.825015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.826787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.838479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 270/540] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.839670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.843797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751d429-0599-4fb5-9edb-4788df7f5bed None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.896956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:58.898063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:58.898354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:58.898635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:58.899406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:58.899723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:58.901150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:58.918008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 271/541] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:58.918983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:58.924079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.954049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08ffeb-8f65-45fe-b1ef-53917f1e2c95 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:58.982776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:58.999727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.000742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.001216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.001681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.002696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=267cd2f0b4d442ee9ab2ee668394472c) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:59.003199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.005385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.011984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 271/542] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/role_assignments?user.id=267cd2f0b4d442ee9ab2ee668394472c&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:59.013033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.068483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ba9d2-680e-498c-a443-3c42e8595872 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.111274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.112524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.113006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.113477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.126608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.128489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.135048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a168de-f590-434e-98fb-a00003aa0f24 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:22:59.156601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:59.157552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:59.158447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:22:59.158447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:22:59.158447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:22:59.158447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:59.163381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 272/543] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:59.165256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.179471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.180674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.181176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.181656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.190261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.192897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:22:59.209189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:59.210084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:22:59.210922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:22:59.213547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 272/544] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:22:59.219568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.237055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:59.294211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:59.313245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a4f38-e979-4087-b648-d3f71388809c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.352421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.353030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.353321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.353683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.354311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:59.354617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.356011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.373480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 273/545] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:59.374721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.394343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195c0cc-c4ff-40fe-85c3-b98db0a1de9d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.468854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.469448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.469738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.470017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.470809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:59.471124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.472585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.505796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 273/546] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:59.511887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.572069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a800-b920-4797-a207-35eca95b3f65 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.603837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:59.634080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.634895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.635176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.635474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.644561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.646080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7bc2d042-d63c-4e56-b076-074722326e87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:22:59.667328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:22:59.668497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:22:59.670233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 274/547] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:22:59.672173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9483504-1bc1-47a5-8d98-a0a05b5cbcff None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.672521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9483504-1bc1-47a5-8d98-a0a05b5cbcff None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.672846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9483504-1bc1-47a5-8d98-a0a05b5cbcff None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.676832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 275/548] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:59.676832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9483504-1bc1-47a5-8d98-a0a05b5cbcff None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.689275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee87f-9602-4682-a678-54fb9a12c6c3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.706797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:59.739208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.740242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.740708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.741790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.743676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=24385618d07d482d9567fe2602fa832a) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:59.744194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.746866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dd119-24b7-46fa-b204-b6998fa75474 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.759316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 274/549] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/role_assignments?user.id=24385618d07d482d9567fe2602fa832a&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:59.759316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.759316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.759316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.792578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:59.793840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:59.846902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9483504-1bc1-47a5-8d98-a0a05b5cbcff None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:59.921393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:59.922079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:59.922360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:59.922633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:59.923595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:22:59.923909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:59.925380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:59.943427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 275/550] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:22:59.947671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 276/551] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:22:59.954622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:59.972531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.012683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.088314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7443ccb-22bb-4a83-aa4a-d9f7171f1774 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.125832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.126653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.126942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.127230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.139506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:00.142887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72f5a-e869-417d-b88f-782069e2974a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:00.148309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:00.149680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:00.151155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 277/552] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:00.155842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.156306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.229038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.229706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.229983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.230256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.257886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.265121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b1b1-b53a-42a0-9967-473e148d7864 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.313315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.314006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.314289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.314563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.315348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:00.315662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:00.317784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:00.334487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 278/553] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:00.340626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.353983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:00.353983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:00.498890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 276/554] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:00.506760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.517324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d97326-539a-4062-b323-fa57a31109f7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.523647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.572371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.573316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.573694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.574064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.585919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:00.587552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:00.609948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:00.610916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:00.611846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:00.611846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:00.611846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:00.611846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:00.615169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 279/555] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:00.620830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.636438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.735596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb8746-0049-42cb-92a8-aa231535bcbe None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.773676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58445d7-a6c7-424e-a2db-46a05440552c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:00.783984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.784745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.785038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.785318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.786214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:00.786530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:00.787975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:00.810797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 280/556] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:00.816486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.822000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:00.822807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:00.823085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:00.823391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:00.847717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:00.849467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:00.881850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:00.883256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:00.884163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:00.884163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:00.884163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:00.884163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:00.884163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 277/557] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:00.898212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:00.935669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:00.973452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f3235-4a00-410c-be34-fc4d12d343c5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.017092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.025046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.025790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.026073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.026346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.039222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.040722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.064350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e0baa-d59d-4730-80ae-9761021e61ad None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:01.069767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:01.070726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:23:01.071780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:01.080855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 281/558] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:01.086920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.107323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.107323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.107323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.107323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.107755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:01.107755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.107755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.117575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 278/559] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:01.123142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.226464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215eec2-c26b-4028-a529-8286ab56e691 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.255820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.263975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.264731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.266415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.266631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.281241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.282611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9422cf85-c6d7-467a-bcbf-ce0cc1c9fc4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:01.305000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:01.305982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:01.306976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 279/560] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:01.307969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816f0b-ab9a-4181-9fc3-9d27fab0ce00 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.308068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816f0b-ab9a-4181-9fc3-9d27fab0ce00 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.311310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad816f0b-ab9a-4181-9fc3-9d27fab0ce00 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.311310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 280/561] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:01.311752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd355da-d705-49f9-b9cc-6f5fc39f878a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.311864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816f0b-ab9a-4181-9fc3-9d27fab0ce00 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.341198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.351652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.352423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.352778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.353125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.354132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:01.354513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.356104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15c1a-5190-49f7-9afb-52829f1d27e2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.376770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 282/562] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:01.378095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.378457 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.378742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.397536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:01.398962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:01.461402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad816f0b-ab9a-4181-9fc3-9d27fab0ce00 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.485476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 283/563] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:01.486176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.510280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.511093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.511509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.511863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.512757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:01.513166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.514706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.527620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 281/564] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:01.528629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.571013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.619405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288b561-171f-49ae-b6b0-a8b04ac7aaa8 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.636844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.654459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8ad8f-43e6-430b-bd36-5c3caf7bc76c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.658093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.658987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.659271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.659568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.671280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.672892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.687698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.688482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.688762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.689030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:01.692199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:01.693434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:01.694345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:01.694345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:01.694345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:01.694345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:01.694345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 284/565] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:01.694617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.704407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-de646e56-7f0b-425b-88b7-096a2d044b4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:01.712725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:01.713635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:01.714479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 282/566] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:01.715722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737c74-0fc8-4bc4-9826-d246dd90719f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.715722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737c74-0fc8-4bc4-9826-d246dd90719f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.716052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65737c74-0fc8-4bc4-9826-d246dd90719f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.719409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 283/567] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:01.719409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65737c74-0fc8-4bc4-9826-d246dd90719f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.808815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39cf27-5d57-4b21-b258-f7a8687ed7c2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.827167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.831907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65737c74-0fc8-4bc4-9826-d246dd90719f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:01.848469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.849152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.849524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.849800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.850620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=826539d8549b40ddacbe997a519691ad) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:01.850928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.852375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.858318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 285/568] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/role_assignments?user.id=826539d8549b40ddacbe997a519691ad&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:01.859092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:01.881514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:01.882720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.883262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.883718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.884784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:01.885254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:01.886849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a6e44-9fda-4c62-b583-c41fd0934819 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:01.901871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 284/569] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:01.903677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:01.904193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:01.904657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:01.910274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:01.928556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:01.930895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:02.006056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9395003-b5da-4608-a5fd-db554166c2e5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.031392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 285/570] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:02.032678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.058059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.058850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.059131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.059519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.060373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:02.060673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.062029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.074278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 286/571] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:02.075129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.141932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.158885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f9b0f-276a-4834-ab41-d3c8d88f4bdf None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.202725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.204249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.204678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.205431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.205925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.214236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e735e9-92a3-4389-9ffe-7859b5d59fc4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.215949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.218808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:02.238049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:02.239186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:02.240376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:02.247929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 286/572] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:02.248281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.261654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.262823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.263319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.263791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.275314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:02.283248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:02.284195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:02.285073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:02.293049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 287/573] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:02.294185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.360608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433c5ac-f903-4f99-9e5c-b0640eab3dd5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.399582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.399801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4debee01-13cc-4582-a143-1594bce06e72 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.400301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.400616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.400896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.414960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.416529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.424876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:02.435568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:02.436473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:02.437413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:02.437413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:02.437413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:02.437413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:02.443443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 287/574] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:02.444723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.447109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.447844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.448127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.448403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.449192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:02.449499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.450925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c24b-4afc-420b-a9d5-0a3c9fb13940 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.464094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 288/575] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:02.465661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0432-4feb-47ba-8948-27342b92cbf4 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.466143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0432-4feb-47ba-8948-27342b92cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.466611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a0432-4feb-47ba-8948-27342b92cbf4 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.468302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 289/576] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:02.469182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0432-4feb-47ba-8948-27342b92cbf4 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.486012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.567311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba5043-860d-4f93-8247-db56a88f8fec None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.576598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a0432-4feb-47ba-8948-27342b92cbf4 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.603908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.605478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.605909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.606602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.607805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:02.608645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.610416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.612865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.613594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.614476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.614971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.615751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:02.616059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.617412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c592ac8-d151-4f8f-81c2-cb0f3b53c87a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.621713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 288/577] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:02.622950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.629345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 290/578] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:02.631112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.631429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.631701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.647935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:02.649126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:02.703731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.719213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 291/579] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:02.720075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.732745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.748091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd6d6f-8817-491e-b8d1-f9dac4f44d3f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.784264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.785445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.785900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.786416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/f2e6dffba870453fab836e0ccece7708/roles/09104180e3694124bf32296605caf033` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.801777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=f2e6dffba870453fab836e0ccece7708, role_id=09104180e3694124bf32296605caf033)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.803212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.827337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 289/580] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:02 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/f2e6dffba870453fab836e0ccece7708/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:02.827337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.849217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c712e-403d-40ed-8a74-244c4423c338 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.900404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.901332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.901924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.902267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.918332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.920745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:02.946471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:02.947652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:02.948608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:02.948608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:02.948608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:02.948608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:02.949434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 292/581] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:02.950391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3016-c4fc-40c4-b0bf-a699ca97a779 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.950521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.967867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.999493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.000352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.001156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.001462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.011869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.013410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.029092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:03.030294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.031405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:03.032369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 290/582] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.032966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.094392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbfa06-03f4-44bd-809b-ef71c4c98780 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.137578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.138795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.139072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.153229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b3e85-0995-4e88-8fa3-d6f54259516d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.156610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:03.165213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.166180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:03.167031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:03.170009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 293/583] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.170992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.192771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.193437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.193716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.193985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.194844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.195226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.196648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.209235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 291/584] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.210132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.243970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.282674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db7f1-c524-4e28-84af-1694b71f5556 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.299998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.318745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e7f0f-c1f9-4259-807c-9dcf8203b18e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.320655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.321573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.322047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.322525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.323596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.324101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.325732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.344388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 294/585] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.345513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.354769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.355643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.355772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.356048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.356793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.357095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.358541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.367249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 292/586] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.367901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.459351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a41ea5-7394-4598-bbcf-a807d7e75f4a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.465400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.474329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef01823-f9ee-41fc-a456-877b9dc92d86 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.495525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.496649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.497142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.497623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.506082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.507784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.510676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.511113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.511438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.511711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.523088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.524573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:03.528511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.529434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:03.530366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:03.539415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 295/587] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.539415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:03.546552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.547487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.548381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:03.548381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:03.548381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.548381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:03.548381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 293/588] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.548665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.584540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.653149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151015e5-ee45-4ef4-9bcd-b793177dbfdd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.661751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a0903-3ebc-4a17-9474-4c31ee403583 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.687703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.688371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.688648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.688919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.689751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.690060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.691875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.703877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.704966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.705438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.705976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.715007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 294/589] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.715007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.715722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.725332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.727238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:03.747004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.748284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.749303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:03.749303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:03.749303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.749303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:03.751161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 296/590] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.752236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.814285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e440efd-1d96-446f-ab44-6bca8ca91408 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.850057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.850610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.852645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.852952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.853216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.853932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.854233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.855620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.868303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 295/591] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.869151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.879384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7e8fc-33a6-4c03-8d8f-2d3385edb9b6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.918611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.920150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.920632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.921121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.932164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.934830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.937535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:03.958332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:03.959272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:03.960199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:03.960199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:03.960199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:03.960199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:03.962231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 297/592] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:03.963412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.972213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af49de-88bd-4f5e-9fc4-e8bc27067e4c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.009930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.010595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.010873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.011144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.011887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:04.012191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.013542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.030972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 296/593] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:04.033002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.093752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12a659-962f-4af5-a51d-2191a34c2bff None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.132075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.132867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.133141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.133408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.139698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7026-752f-43c3-a3cf-eeb0cd411ac7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.140312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.141725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.143537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-ee6318c6-e6ec-48cd-93e3-50a37976c577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:04.157336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:04.158330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:04.159220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 298/594] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fc0f6-fae3-424d-9972-072012679279 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fc0f6-fae3-424d-9972-072012679279 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fc0f6-fae3-424d-9972-072012679279 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 299/595] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:04.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fc0f6-fae3-424d-9972-072012679279 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.173535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.186252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.186920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.187204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.187552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.188256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:04.188557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.190032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.202283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 297/596] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:04.203057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.282663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fc0f6-fae3-424d-9972-072012679279 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.310158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90551775-8d76-4183-b8e0-f61f50bf6387 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.318867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.319896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.320175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.320446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.327753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.329173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-27e51965-0c7f-4bb3-a4f1-51cded5cd63e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:04.343424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:04.344381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:04.345322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 300/597] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:04.346494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.346494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.346494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.350898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.351594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.351874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.352150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.352933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:04.353247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.354659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.364137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:04.365613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:04.369173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 298/598] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:04.369826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.421537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.428265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.455783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 301/599] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:04.456766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.496887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0377d5-5982-42fd-b106-b1f0e51768d4 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.534179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.535442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.535442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.536257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.543086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.544573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:04.558710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:04.559705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:04.560592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 299/600] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:04.561092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.575608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1490890-40b7-45e1-a9e6-61709f6c3dd5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.610395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.611228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.611644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.612016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.612958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:04.613371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.615554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.633309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 302/601] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:04.634141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.683641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb2ad-6094-4511-8d96-7b1d8970baad None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.684168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.687234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.721435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.722329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.722614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.723578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/09104180e3694124bf32296605caf033` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.739313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=0364cf432f5b44a9a62f08c4243fff5e, role_id=09104180e3694124bf32296605caf033)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.740789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.742027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3529-415d-4ca4-a8a7-0a05dca85278 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.763326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 300/602] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:04 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:04.763326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.793269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.794390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.795068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.795823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.811077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:04.822794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:04.824425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:04.827856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:04.829220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 303/603] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:04.829220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.898119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b80d-533c-4a1b-813b-b5004d63065b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.943139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:04.945209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:04.945854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:04.946144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:04.946945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:04.947255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:04.948714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:04.962527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 301/604] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:04.963189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:04.978823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb23da-81d8-4c24-a221-1d16cfa01a7d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:04.986025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:04.987926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.020177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.021106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.021391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.021666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.033286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.034742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:05.054044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:05.055063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 304/605] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:05.056260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.086115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765623fb-0cc8-47b7-9173-74481a5a90f3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.122124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.123106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.123603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.124073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.125117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:05.125729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.127172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732dce9-2a35-4264-9b49-b2df841ea135 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.140504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 302/606] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:05.140504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc52f-932c-4ff2-829d-844f9f5d3aeb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.140504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc52f-932c-4ff2-829d-844f9f5d3aeb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.140504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcc52f-932c-4ff2-829d-844f9f5d3aeb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.140504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 303/607] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:05.141400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcc52f-932c-4ff2-829d-844f9f5d3aeb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.169185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086f3d-87af-4187-aff6-cd6a7f0d81aa None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.205241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.206140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.206422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.206719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/267cd2f0b4d442ee9ab2ee668394472c/roles/09104180e3694124bf32296605caf033` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.223690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=267cd2f0b4d442ee9ab2ee668394472c, role_id=09104180e3694124bf32296605caf033)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.225748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.226227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.241017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 305/608] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:05 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/267cd2f0b4d442ee9ab2ee668394472c/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:05.248587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.253882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcc52f-932c-4ff2-829d-844f9f5d3aeb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.265922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.297169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.297857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.298135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.298414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.299231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:05.299595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.301079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fb218-25e9-4b4e-a1df-8c0589c2c278 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.315558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 304/609] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:05.317090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.317448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.317730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.337858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:05.339448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:05.393964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edb054-4abd-48ae-9b61-d6ab89899f7d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.426904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 305/610] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:05.428144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.436459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.437269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.437551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.438327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.456867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.458388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:05.482296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:05.483516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:05.484523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:05.484523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:05.484523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:05.484523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:05.484523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 306/611] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:05.485475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.510975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.556922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b672c-e7cb-4b6b-b197-bfeb5199868c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.557999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.605414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.606380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.606666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.606944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.610433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de740b4e-e810-478a-b02e-1c891cada71b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.616937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.618391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:05.638735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:05.639740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:05.640644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 306/612] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:05.641158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.659611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.660952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.661234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.661510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.675605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:05.684467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:05.685845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:05.686720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:05.691225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 307/613] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:05.692363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.777822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c696-021d-4a18-b3ee-b55324b0e16c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.807881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.819586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745bce-8869-471d-996a-82190f93c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.820672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745bce-8869-471d-996a-82190f93c89d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.821182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745bce-8869-471d-996a-82190f93c89d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.821707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745bce-8869-471d-996a-82190f93c89d None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.822752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745bce-8869-471d-996a-82190f93c89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:05.823336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745bce-8869-471d-996a-82190f93c89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.824948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745bce-8869-471d-996a-82190f93c89d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.825741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0b6-f0e8-4bd9-b90d-c848877b1b29 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:05.843033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 307/614] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:05.844228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745bce-8869-471d-996a-82190f93c89d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.850900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:05.869316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:05.869916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:05.870200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:05.870478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:05.871269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:05.871606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:05.873564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:05.891667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 308/615] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:05.892840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:05.969331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745bce-8869-471d-996a-82190f93c89d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.007033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.008177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.008305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb062-95db-423d-9dc3-bc2c3382be79 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.008787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.009271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.010334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:06.010910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.012618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.022748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 308/616] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:06.023623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.051546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.052567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.052944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.053225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.061227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.062656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.075000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:06.080385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:06.081537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:06.083545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:06.085896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 309/617] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:06.086914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.132825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.133148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cc8eb-f503-405f-abba-bd6a1d8e8f70 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.169658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.170856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.171140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.171812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/24385618d07d482d9567fe2602fa832a/roles/09104180e3694124bf32296605caf033` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.187692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=24385618d07d482d9567fe2602fa832a, role_id=09104180e3694124bf32296605caf033)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.189615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.200022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd308a2-a4a9-4c63-89a2-634b9cdd6849 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.211377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 309/618] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:06 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/24385618d07d482d9567fe2602fa832a/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:06.212766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.266327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.267510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.268716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.269304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.290328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.298044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:06.324354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:06.325504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:06.326765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:06.326765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:06.326765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:06.326765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:06.332170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 310/619] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:06.338631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.406018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc85bc-757c-460c-823c-7ac20dd3c555 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.439701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.466364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.467063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.467392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.467672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.468524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:06.468834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.470258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.493858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.494382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 310/620] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:06.500548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.505110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98ea08-0656-4a4d-ab8f-866343980087 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.541461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.542565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.543089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.543596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.544575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:06.545355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.546925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.560351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 311/621] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:06.567317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.665535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c1969-9b66-488c-8ec2-81ae43e92c76 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.719423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f71ee-d4f8-4587-b2bf-c1dc734ef59d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.757782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.758578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.758858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.759130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.760471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.761275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.761558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.761833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.767193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.768663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:06.784943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:06.786253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:06.787235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 312/622] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:06.787896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.791311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.794957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.812321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:06.835693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:06.837826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:06.839981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:06.839981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:06.839981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:06.839981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:06.839981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 311/623] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:06.848481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:06.871665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:06.918608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc61d2-f810-4a92-8d53-844a4b83d58a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:06.957648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:06.958417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:06.958702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:06.958978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:06.959866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:06.960176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:06.962131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:06.982707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 313/624] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:06.998243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.043168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9f9d1-711f-45a1-967b-0e6bc05d86fb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.078926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.079639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.079926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.080201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.080932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:07.081237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.083733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ceb40f-fb82-4d5f-8967-3432c055cfe1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.096364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 312/625] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:07.098041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84eeec-9706-4a1e-9d30-dc76e916ecfa None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.098526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84eeec-9706-4a1e-9d30-dc76e916ecfa None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.098888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84eeec-9706-4a1e-9d30-dc76e916ecfa None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.104264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 313/626] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:07.104264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84eeec-9706-4a1e-9d30-dc76e916ecfa None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.117088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.150259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b7e5d-10d0-48dd-85cc-3c5e20774fc7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.185686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.186686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.187216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.187590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.187935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.217099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84eeec-9706-4a1e-9d30-dc76e916ecfa None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.253495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.254863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.255144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.255460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.261287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.263061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.263174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.264743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9ad0c118-0a8e-4e41-bb2d-67694ee993a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:07.280989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:07.281977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:07.282966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 314/627] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:07.284036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.284300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.284480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.302570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:07.304138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:07.340329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.383310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 315/628] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:07.383310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.384271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 314/629] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:23:07.390105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.446040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.485747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267f2ef-676c-4c88-b176-15edb8070141 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.508058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55041b4d-3f66-44dd-a60e-07dab402c832 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.521298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.522146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.522592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.523027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.524011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:07.524499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.525927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.543501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 316/630] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:07.544371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.544588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.545639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.545919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.546189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.558553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.560044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-75672e4d-0a53-4d66-bfe1-d93a114b0487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:07.578775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:07.579893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:07.580832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:07.580832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:07.580832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:07.580832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:07.580832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 315/631] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:07.582732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c106c-850f-4766-9535-142422cc1517 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.582732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c106c-850f-4766-9535-142422cc1517 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.582732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c106c-850f-4766-9535-142422cc1517 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.583969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 316/632] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:07.584675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c106c-850f-4766-9535-142422cc1517 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.611709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.655242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47f9d7-f651-42ea-b581-e5a68580b6d8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.673616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.693348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.694078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.694363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.694640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.696689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c106c-850f-4766-9535-142422cc1517 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.705153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-f9d51ede-43ad-4688-ba22-353106eb26a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:07.714548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:07.715542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:07.716388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 317/633] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:07.717763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5469d-e322-404b-b4da-c379b716766d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.717763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5469d-e322-404b-b4da-c379b716766d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.717896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5469d-e322-404b-b4da-c379b716766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.735004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fee5469d-e322-404b-b4da-c379b716766d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:07.736286 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fee5469d-e322-404b-b4da-c379b716766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:07.737648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.738823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.739323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.739446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/826539d8549b40ddacbe997a519691ad/roles/09104180e3694124bf32296605caf033` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.755551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=826539d8549b40ddacbe997a519691ad, role_id=09104180e3694124bf32296605caf033)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.757012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.772152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 317/634] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:07 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/826539d8549b40ddacbe997a519691ad/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:07.773330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.839762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 318/635] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:07.840850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5469d-e322-404b-b4da-c379b716766d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.874867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.915978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea549297-7f87-4456-a483-d55f1ef68728 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.966179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.967495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.967495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.967495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.968781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:07.968781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.970701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.975722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5469d-e322-404b-b4da-c379b716766d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.986028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.986202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 318/636] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:07.987056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.036248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.036929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.037208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.037480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.038326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:08.039199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.040597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.054218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 319/637] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:08.055333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.124113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cca897-13dc-4f64-b15b-e111c9624e8b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.157671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.164585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.165785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.166289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.166760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.179604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:08.180180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e8c-ce2b-49a8-87ff-9efc9fa2f6f1 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:08.189812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:08.190829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:08.192252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:08.194992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 319/638] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:08.196181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.220096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.220818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.221101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.221374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.231538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.233812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:08.254328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:08.255854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:08.256937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:08.259934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 320/639] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:08.261171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.306580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.320760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d4aa5-fc10-4c99-ba4e-0bf03f285fc7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.357809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.358763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.359141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.359678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.371740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.373354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:08.392753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:08.393922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:08.394933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:08.394933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:08.394933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:08.394933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:08.397175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5144560-6a67-4eaa-9578-6ea2f7f9b20b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.400313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 320/640] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:08.400581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.434100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.435141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.435670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.436139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.437084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:08.438056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.439668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.452326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 321/641] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:08.453521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.454866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.522641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb799883-565d-460d-bdf8-6e7cc53f6b2d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.555358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.565240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.566143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.567146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.573702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.576148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fced545-1182-4e8f-8043-ef8b769fb391 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.577123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:08.577589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.579483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.614957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 321/642] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:08.616099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.618664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.619360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.619653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.619930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.620651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:08.620959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.622318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a26902-bc24-41c9-9c4d-b1db82d63a45 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.639369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 322/643] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:08.639369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbbb94-9925-425e-93cd-47b5dc23c512 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.639369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbbb94-9925-425e-93cd-47b5dc23c512 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.639369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbbb94-9925-425e-93cd-47b5dc23c512 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.642787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 323/644] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:08.643645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbbb94-9925-425e-93cd-47b5dc23c512 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.746827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2f11-a9cb-4135-83d9-10a3cee68d4e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.750693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.763724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbbb94-9925-425e-93cd-47b5dc23c512 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.787429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.788838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.789298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.789742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.804329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.805794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.808130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.808630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.808903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.809177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.809996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=f2e6dffba870453fab836e0ccece7708) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:08.810308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.811763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.818295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 324/645] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/role_assignments?user.id=f2e6dffba870453fab836e0ccece7708&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-00344532-5adf-4ea5-a1d7-81f640403dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:08.825001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:08.825942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:08.827049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:08.827049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:08.827049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:08.827049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:08.831258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 322/646] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:08.832123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.833007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.833298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.833563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.852487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:08.853154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:08.901332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:08.944802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 323/647] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:08.944802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.080455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.086835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44317342-34de-4f61-8bf7-5523ac15a156 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.104132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c094b7-c609-4f62-a243-59cfc38451cb None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.132762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.133557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.133838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.134117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.141571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.143009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.151322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.151322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.151322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.151659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.155122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:09.155122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.155122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:09.161293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:09.162433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:09.163671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 325/648] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:09.171378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.174141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 324/649] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:09.175124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.250176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.276218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075bc9-259e-4f1d-899e-ff3557c456db None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.287082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbadc0-99d3-4d11-aaf5-fac2b11deb94 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.312778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.313584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.313865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.314141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.324554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.325365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.325776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.325890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:09.326235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.327182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:09.327629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.329160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:09.334097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:09.335192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:09.336264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:09.339529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 326/650] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:09.340421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.352043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 325/651] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:09.353132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.360353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.480506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cff5a-143f-492e-8195-2c08c2f17c3b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.497084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb63563-e49f-402e-aef7-eaccc468db7f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.522884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.524374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.525122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.525939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.530074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.537848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.538785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.539089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.539398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.540157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:09.540289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.540489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.541833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.542340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.556847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 326/652] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:09.557737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:09.560621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:09.561591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:09.562545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:09.571073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 327/653] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:09.572119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.638139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.680535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4ad4-059d-4228-bc85-15497a188ece None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.696249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413e581-e417-41e4-b40c-b935df32fd75 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.718787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.719968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.720576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.721080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.731740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.732398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.732674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.732942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.744930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.746533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:09.765322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:09.766290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:09.767184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:09.767184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:09.767184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:09.767184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:09.771645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 328/654] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:09.772486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.803801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.806152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.824278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.877418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:09.878303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0660f8d-7038-463c-be54-cb60642630af None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:09.913756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:09.914366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.914635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.914871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.919467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 327/655] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:09.919632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:09.919632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:09.919800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:09.920130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618d462-fa47-499c-9bee-87a15287f678 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:09.936795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 329/656] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:09.938079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c2e1-7d61-41f7-a229-919147679e68 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:09.938362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c2e1-7d61-41f7-a229-919147679e68 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:09.938635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34c2e1-7d61-41f7-a229-919147679e68 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:09.940104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 330/657] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:09.940647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34c2e1-7d61-41f7-a229-919147679e68 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.031723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257a152-44f2-436d-bc86-8ab1485029d3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.057230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34c2e1-7d61-41f7-a229-919147679e68 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.065256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.078277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.078944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.079224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.079529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.080259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:10.080562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.081958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6147b6-2a31-4a04-a7a7-bdf0b1be53c5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.094423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 328/658] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:10.095693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.095981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.096258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.100260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.101123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.101398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.101837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.113702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.114483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:10.115218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.115774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:10.134738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:10.135908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 331/659] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:10.136842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.145289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.196486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 329/660] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:10.197302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.258379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d7f4-b1bf-4461-88b7-d7fd08fb78cb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.294524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.295458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.295748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.296021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.300058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e3a-60af-485d-bbcd-9ba5f1160a2a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.323654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.341936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.342730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.343014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.343314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.351139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.352268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:10.367371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:10.368342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:23:10.369885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:10.370452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.370528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 330/661] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:10.370528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.374059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.388858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.480303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d273a9d-e971-4126-a2b6-d66005de458a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.490886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 332/662] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:10.491798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.522229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.523773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.524237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.524812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.525866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:10.527095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.528676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.545581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 331/663] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:10.545581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.556923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.600439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae12ee6-e7ed-490b-8a93-3f63ff76b3e2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.628342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.636502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.637297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.637803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.637925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.649674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:10.653565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c29864-433b-44db-8c5b-1ea6c3bf52d8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:10.657456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:10.658392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:10.659253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 333/664] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:10.659996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.695820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.696664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.697272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.697557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.698394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:10.698708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.700159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.726128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 332/665] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:10.726128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.770605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c6712-6122-4de7-aa4c-cb0d34b40fde None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.788952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.808068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.809117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.809401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.809677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.820978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.822425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.836893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57c278-f73d-4f24-b563-44d2b465859f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:10.839938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:10.840829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 334/666] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:10.842351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.875109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.875934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.876217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.876492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.878464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:10.885104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.886954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:10.901711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:10.902645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 333/667] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:10.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:10.956540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a0202-adcf-4447-92c6-90a5180d6425 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:10.992768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:10.993439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:10.993715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:10.993982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:10.994720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:10.995019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:10.996400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.009762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 335/668] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.009762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.019296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cdb155-597b-48ff-8ff3-212a73699def None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.038653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.055435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.056081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.056357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.056625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.106488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036c8b1-7ed9-433a-a841-f8336d24c540 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.117301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.125547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.127114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.139037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.139725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.140007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.140282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.140995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.141611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.142643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.154770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 336/669] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.155735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.228781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 334/670] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:11.229541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.247935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.257407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8e868-1538-4c6f-b71a-3ef1810db167 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.290052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.290645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.290927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.291202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.292016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.292327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.293750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25ca6a-cbbf-44d2-92da-07fdd36c2b6a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.305630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 337/671] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.306852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a52-4b42-4ae9-bbbf-adeb04b63624 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.307141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a52-4b42-4ae9-bbbf-adeb04b63624 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.307452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd26a52-4b42-4ae9-bbbf-adeb04b63624 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.308896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 338/672] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:11.309406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd26a52-4b42-4ae9-bbbf-adeb04b63624 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.326415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.333092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5b9a0-d6ac-47e6-a14e-cc8a5a067dcb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.365878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.366662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.366941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.367214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.376983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.378403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:11.396016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:11.396927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 335/673] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:11.397924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.413647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd26a52-4b42-4ae9-bbbf-adeb04b63624 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.448953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.449748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.450027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.450321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.456840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.458830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:11.472624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:11.473650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 339/674] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:11.474576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.490086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.499815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bec5ae-3a91-4ddd-989b-33d2e67c4748 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.534719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.535695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.535987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.536271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.546725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.548148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.555555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-65fa03a4-1ca6-4818-9bb4-d4ad065007e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:11.567811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:11.568745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 336/675] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:11.570223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.570646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.570738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.574696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f28e91-45d5-4a2c-b246-e9326f336ca5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.587112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:11.588338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:11.608832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.609484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.609768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.610045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.610872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.611191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.612646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.618501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 340/676] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.619274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.659649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 337/677] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:11.661428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.718389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.752681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0d889-b4dd-4d3c-9713-b38f6117c297 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.789052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfc669-6de8-4a6e-acf1-52d431355b9e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.792536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.793203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.794180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.794180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.794464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.795494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.796962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.809683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 341/678] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.810718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.821526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.830922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.832039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.832039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.832769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.834749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.834749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.837361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.850466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 338/679] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.851133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.927268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee6b5e-a644-4258-889b-580acdc2e992 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.959465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840c48-b556-4913-b727-e16259ec38de None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:11.960821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.961493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.961772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.962045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:11.962770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:11.963068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:11.964513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:11.966827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:11.981847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 342/680] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:11.982795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:11.997341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:11.998414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:11.998894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:11.999376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.011046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.012760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:12.030868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:12.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:12.032999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:12.032999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:12.032999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:12.032999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:12.034881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 339/681] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:12.036409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.083412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.109407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f56f7-e299-470a-bdb0-4f54ff0798b0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.146548 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7e558-a2dd-4e21-9d59-1f0187fef784 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.148718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.149510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.149789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.150057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.157323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.158755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:12.173839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:12.174829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 343/682] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:12.176527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.185651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.186497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.186925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.187405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.188426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=267cd2f0b4d442ee9ab2ee668394472c) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:12.188884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.190478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.198260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 340/683] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/role_assignments?user.id=267cd2f0b4d442ee9ab2ee668394472c&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:12.199254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.263797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.326853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2dc22e-b8da-4ea3-a569-6003216b17bc None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.330358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.331501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f6f04-3892-40e7-adde-1ecbf48dc4cb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.373404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.373994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.374275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.374550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.375350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:12.375626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.377011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.392385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 341/684] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:12.393247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.393377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.394433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.394815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.395152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.404186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.406091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:12.425540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:12.427393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:12.428551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:12.429492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 344/685] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:12.430524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.517860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c78f-deb4-4fca-9227-d4a827b12d79 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.557682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.558267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.558549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.558825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.559968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:12.560283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.561821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.568086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b846a17-bd36-4e38-b155-21973cd9234d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.581233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 342/686] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:12.582816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.589271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.610944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.616615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.617822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.618517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.618986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.620435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.622063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.648662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 345/687] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:23:12 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:12.650033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.701490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ba618-6b0d-4216-bcd4-a753b960c406 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.743135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.744006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.744383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.744757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.745942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:12.747127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.748182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.762315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 343/688] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:12.768129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.781461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80101e-6c6a-4aa6-9719-81b80a591c19 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.822074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.822984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.823468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.823957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.866608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.874670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:12.891389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f372864-9a61-4400-b183-65be0f257c42 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:12.915740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.917145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.930008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:12.931047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.931572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.932068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.933109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=24385618d07d482d9567fe2602fa832a) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:12.933633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:12.935391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:12.942888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 344/689] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/role_assignments?user.id=24385618d07d482d9567fe2602fa832a&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:13.033737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 346/690] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:13.039900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:13.167022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853445-9a17-4820-9c5e-51b6bf602fa1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:13.206274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:13.207510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.207979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.208434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.220952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:13.222431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:13.229821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:13.243155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:13.244145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:13.245048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:13.245048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:13.245048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:13.245048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:13.245048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 345/691] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:13.250540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:13.325991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:13.361640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932c065-2e07-43bc-873c-9bced34cae7b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:13.408545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:13.409221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.409505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.409791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.410770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:13.411090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:13.412673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:13.425062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 347/692] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:13.431269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:13.573330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b52eba-ba7b-4a9e-8097-92aafde8e2a8 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:13.619559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:13.620358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.620642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.620918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.632436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:13.633906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-88cb0cdb-9c1b-4db0-94fe-eaee1f184694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:13.652612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:13.653560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:13.654467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 346/693] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:13.660089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:13.768841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb76fb9-c3f0-45a5-ae3c-bba48dc0fa47 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.769169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb76fb9-c3f0-45a5-ae3c-bba48dc0fa47 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.769397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb76fb9-c3f0-45a5-ae3c-bba48dc0fa47 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.771596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 347/694] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:13.772160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f033fd-24bd-4b38-a497-5ab2251c360c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:13.779494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.779494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.779494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.801851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:13.803064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:13.825197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:13.833631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:13.834305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.834593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.834874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.835944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:13.836259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:13.840293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:13.861636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 348/695] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:13.868981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:13.907992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:13.924455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 348/696] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:23:13.976805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.035143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d4ce4-ce5f-4b08-9e25-1f1cd7ff99e3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:14.100883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:14.101914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.102452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.102952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.104099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=826539d8549b40ddacbe997a519691ad) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:14.104622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:14.106254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:14.119094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ad5f-38cf-47ba-92cd-23b814ad0894 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:14.120353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 349/697] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/role_assignments?user.id=826539d8549b40ddacbe997a519691ad&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:14.162850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:14.163712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.163998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.164276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.188599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:14.197107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:14.198175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:14.199056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 349/698] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:14.204610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.299436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:14.378257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b5907-1cdc-48d3-bdf3-45aaab38b17f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:14.416395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:14.417064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.417342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.417613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.418353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:14.418655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:14.420304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:14.433182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 350/699] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:14.434599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.488807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:14.547822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19feaf-e4dd-490c-b40c-3f59b197137e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:14.584290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:14.584998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.585275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.585549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.599479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:14.600923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-8b40bdd1-77f7-4751-a818-a69d57bc24fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:14.624641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:14.625618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:14.626523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:14.626523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:14.626523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:14.626523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:14.626523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 350/700] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:14.632154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.721314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa46a-fd5d-48e0-8288-bf3e99f46680 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.721644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa46a-fd5d-48e0-8288-bf3e99f46680 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.721866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acaa46a-fd5d-48e0-8288-bf3e99f46680 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.724560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 351/701] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:14.728647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.728932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.729208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.758908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:14.760125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:14.767549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef34834-073d-458d-b9ae-9343c546aec3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:14.811275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:14.839932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 352/702] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:14.851011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:14.851721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:14.852010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:14.852291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:14.853026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:14.853336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:14.854716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:14.861097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.883967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 351/703] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:14.890224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:14.919557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.026394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aabd86-6d70-4e1e-b55c-f6b5a31a0016 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.067615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.068795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.069281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.069774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.070669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.072841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.075400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30989-0009-4a4f-ae6d-27ce1480f616 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:15.097338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:15.098921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:15.099833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:15.102771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 353/704] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jul 20 09:23:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:15.109704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.124638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.125433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.125709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.125979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.133906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.135594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:15.150768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:15.151825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:15.152794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:15.155575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 352/705] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:15.163616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.200929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.287331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0a44e-798f-4d10-bc92-517af1b9b1be None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.291722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.307611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516bdda-1192-4126-a44d-c3826dc3c95f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.324012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.324890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.325073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.325332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.326206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:15.326513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.331309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.351506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 353/706] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:15.353615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.381124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.382109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.382575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.383028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.383945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:15.384435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.386002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.399458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 354/707] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:15.404608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.465657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.473179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cc3eb-6d80-4751-9cd3-f18a00b33bbc None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.510767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.511454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.511731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.512003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.590666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.592287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.621867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44e812-8b1a-4b36-a990-60e48fed32ea None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.665677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.666577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.666859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.667730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.668436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:15.669016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.670904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8032c-23be-4422-a692-21fe2cf3a020 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.682831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 355/708] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:15.684534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6462-6eda-4f69-afb3-278bd872ac40 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.685305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6462-6eda-4f69-afb3-278bd872ac40 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.685583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e6462-6eda-4f69-afb3-278bd872ac40 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.687593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 356/709] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:15.698333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.698665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.698887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.710477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.727476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 354/710] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:15.732405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:15.733863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:15.736931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.756264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:15.849212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 357/711] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:15.875104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.882707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889267-036a-412a-9b0c-cf0bf465ade5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:15.922305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:15.923991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:15.924521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:15.925450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:15.939773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:15.942247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:15.963559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:15.964557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:15.965500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:15.965500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:15.965500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:15.965500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:15.969458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 355/712] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:15.970704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:15.996588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbdd9a-bdd8-4487-836a-34f8de3dc49c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.034723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.036755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.037265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.037730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.049481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c273fb66-f8bb-4508-b1ba-b4c7e36b538c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:16.058596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:16.059641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:16.060566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:16.061285 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 358/713] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:16.061941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1c53-824c-486d-867f-efa127bc5531 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.062226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1c53-824c-486d-867f-efa127bc5531 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.062498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c1c53-824c-486d-867f-efa127bc5531 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.067338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 359/714] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:16.070205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.070496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.070775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.077960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.088200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:16.089398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:16.099872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.150304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be012-650b-4924-982c-d8ddb0476dcb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.175407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 360/715] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:23:16.177276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.195116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.196160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.196672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.197149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.198076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.198634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.200283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.218110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 356/716] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.219539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.302102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67600a-f8b6-4781-b85d-a174656a2ced None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.339026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd3035-630c-484e-b889-cb6279196c55 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.342575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.342779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.342987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.343263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.344053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.344363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.345966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.350983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.359334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 361/717] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.359334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.361857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.388363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.389542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.390024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.390622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] PATH_INFO: `/v3/services/network` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.391622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.393189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:16.408746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:16.409767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:16.410818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:16.413832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 357/718] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:16.414890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.473100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898419-7cfd-441e-a46b-b950173302c0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.509158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.509967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.510254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.510532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.517724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.519213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:16.534408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:16.535709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:16.536633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 362/719] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:16.537136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.544401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52894d49-ef5a-46c8-85dc-33d777c5566e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.582738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.584079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.584568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.585115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.597048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.599280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.601815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:16.621697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:16.622655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:16.623967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:16.623967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:16.623967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:16.623967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:16.630466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 358/720] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:16.631595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.641496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf76d-d9cd-40aa-af8e-e688413aa877 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.661469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.678389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.679066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.679377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.679658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.680368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.680676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.682041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.691176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 363/721] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.692192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.749002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98169e-f04a-42ab-941a-b09a47b59cd5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.784219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.784878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.785160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.785677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.786471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.786784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.788267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.795535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7ec55-72a7-4d2f-9190-ebbb40ef21b5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.800624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 359/722] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.801822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.823316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.833505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.834171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.834655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.834768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.835470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.835776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.837136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b8b46-5f31-46c1-8cea-51a43ba4b88e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.853673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 364/723] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.854822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e019-3074-4f8c-be3c-d69da6f4c825 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.855154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e019-3074-4f8c-be3c-d69da6f4c825 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.855467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658e019-3074-4f8c-be3c-d69da6f4c825 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.859360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 365/724] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:16.859360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e019-3074-4f8c-be3c-d69da6f4c825 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.894596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:16.911422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d52f1-e15f-45cf-9c28-69220ece56e0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:16.947188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:16.947795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:16.948074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:16.948346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:16.949041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:16.949342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:16.950699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:16.956538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 360/725] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/services?type=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:16.957781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:16.988064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658e019-3074-4f8c-be3c-d69da6f4c825 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.038582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.040608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.041217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.041743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.043022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:17.043552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.047176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6302-fd3a-4fe2-9d3a-c65e77cbcd06 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.056196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 366/726] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:17.057848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.058316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.058741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.080710 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:17.082317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:17.113094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e63fc2-c0c0-4314-9c17-cb94c9a471a0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.148178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:17.183317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 367/727] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:17.185856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.186958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.187256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.187660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.200067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.201454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.211244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:17.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:17.229065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:17.230021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:17.232827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 361/728] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:17.241165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:17.257303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:17.328985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8193c-bf5d-4d6b-88db-b9f224c4b288 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.364969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.365779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.366061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.366335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.377046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:17.397355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:17.398365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:17.399239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:17.402029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 368/729] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:17.407789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:17.418634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c65af1-e3cb-4128-8dd5-dccb639d2730 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.453557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.454161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.454449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.454724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.455590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:17.455901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.457403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.477755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 362/730] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:17.489259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:17.507899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:17.570157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123555ea-8a5e-4309-b48b-f65d35db710e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.606307 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.607334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.607623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.607907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.609728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:17.610041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.614827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.632384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 369/731] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:17.637533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:17.639636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:17.708407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe5e33-095c-452b-aaec-588b6bc36452 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.773963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.774620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.774900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.775173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.822347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242a09f-626a-456e-8689-c82958a2904d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:17.827030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:17.888461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:17.889267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:17.889545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:17.889818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:17.893468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.895564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929dfbf-4c58-452e-9991-1f84145b97c4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.911181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:17.912681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:17.944170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:17.945123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:17.946032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:17.946032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:17.946032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:17.946032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:17.946032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 370/732] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:17.952306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.002225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.040738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 363/733] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:18.046519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ebe6-5569-4f3d-8012-a52a3e1259ba None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.046519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ebe6-5569-4f3d-8012-a52a3e1259ba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.046519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1321ebe6-5569-4f3d-8012-a52a3e1259ba None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.048321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 364/734] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:18.049107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321ebe6-5569-4f3d-8012-a52a3e1259ba None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.077501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6175-1ed1-41f7-9990-79eadefc2790 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.114871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.115936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.116389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.117040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.118059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:18.118568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.120114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73210203-11f6-442d-bad9-ed32c67d8876 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.136734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 371/735] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:18.138471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.138865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.139239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.156711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:18.157995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:18.158538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1321ebe6-5569-4f3d-8012-a52a3e1259ba None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.169669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.198507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.199630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.200100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.200585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.211759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.213540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.235329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 372/736] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:23:18.235329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:18.237202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:18.238095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:18.239168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:18.239168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:18.239168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:18.239168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:18.241501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 365/737] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:18.248329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.279460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.347268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6726b0-bad9-4d5c-82ce-1a24e2f40ad3 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.370514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336131f4-9ebf-49bb-a8cf-af0d2ed49a1b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.385391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.387176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.387574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.387920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.399117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.401255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.409958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.411304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.411677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.412020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.412826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:18.413204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.414632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:18.421160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:18.422298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:18.423871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:18.425034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 373/738] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:18.425034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.429802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 366/739] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:18.430754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.436731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.521265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.546218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e014-aa61-4533-bb03-abe86ca50080 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.563580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3db2d-7dd2-4da5-bd10-d18ab750706d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.582106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.583256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.583788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.584271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:18.585886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.587616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.605713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 367/740] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:18.606632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.607089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.607318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.607628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.607905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.608652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.610074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.635408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 374/741] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:18 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:18.636734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.782182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.817631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0370c-3ada-4ef3-b798-74bb91653f3a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.821580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec4498-6371-469d-bf66-35d437ae4cb6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:18.866271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.868262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.869040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:18.869870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:18.870330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.870800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.871249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:18.871721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:18.874670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:18.881166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.883097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:18.902126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:18.903095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:18.904419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 368/742] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:18.908831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:18.967571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:18.970880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.034208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.038809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eee8ca-3ef8-4cee-88a6-535a0d9fd7eb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.082890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 375/743] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:23:19.082890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.084545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.085445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.085957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.086424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.087450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:19.087955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.089700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.112220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 369/744] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:19.118647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.187941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f96f44-ce8b-41a9-89c3-80cdbb64e96e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.212810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.227447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.227447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.227737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.228055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.252320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.253790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:19.285891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:19.286923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:19.288469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:19.288469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:19.288469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:19.288469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:19.288469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 376/745] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:19.300279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.300441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5c14e-6724-4106-99fc-1d1f55fe7fbf None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.320993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.346139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.348073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.348542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.349682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.350893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=1ab2924b8a9d41a7a6b74a73a7b809bd) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:19.352664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.355073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc52de7-b15f-48ba-a760-9053b9888954 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.369600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 370/746] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/role_assignments?user.id=1ab2924b8a9d41a7a6b74a73a7b809bd&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:19.407690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bfc58-b365-4f74-bb21-0496e0c3ca97 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.407997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bfc58-b365-4f74-bb21-0496e0c3ca97 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.408410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bfc58-b365-4f74-bb21-0496e0c3ca97 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.415181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 371/747] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:19.416942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837fee-4bc7-4cba-b47b-dfb0a97ed0fd None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.417227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837fee-4bc7-4cba-b47b-dfb0a97ed0fd None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.417500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94837fee-4bc7-4cba-b47b-dfb0a97ed0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.454197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-94837fee-4bc7-4cba-b47b-dfb0a97ed0fd None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:19.463325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cc6d9-c99b-446c-a784-2be57f46b88e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.464075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-94837fee-4bc7-4cba-b47b-dfb0a97ed0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:19.502967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.503678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.503981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.504794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.505545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:19.505852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.507322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.533531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 377/748] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:19.542065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.623415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 372/749] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:23:19.623415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283cdd4-3292-4627-8161-ce0cb32d09d4 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.623415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283cdd4-3292-4627-8161-ce0cb32d09d4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.623415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283cdd4-3292-4627-8161-ce0cb32d09d4 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.623899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 373/750] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:19.631056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.631848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.632621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.645863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.655219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:19.657323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:19.672732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c5c24-3ef3-4d52-b8ae-bfc0e7c360ad None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.713290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.714132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.714413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.714680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.718991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.725236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.726693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:19.741229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:19.742163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:19.743073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 378/751] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:19.744023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.750674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 374/752] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 09:23:19.751935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.863235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca6990-11ba-4726-a83f-27d4b2b1128d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.868959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132440f-ccf2-4c97-a3e6-54795480d1a7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:19.902993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.903610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:19.903775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.903996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.904266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.905216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:19.905715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:19.906142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:19.906617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:19.908628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:19.909240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:19.911859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:19.916169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:19.924085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:19.925042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:19.925963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 379/753] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:19.926660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.929962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 375/754] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:19.937120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:19.993433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:20.050116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee4278-0969-4f29-ae91-3b8ec543ca51 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.068201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ffdc0-7aa8-4f07-ab4c-f4a12f37faec None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.096065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.097430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.098288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.098711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.100030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.101416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.109192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.110271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.110883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.111382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.112451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=826539d8549b40ddacbe997a519691ad) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:20.112987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.115398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.123316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 380/755] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:19 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:20.123316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:20.130833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 376/756] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:19 2022] GET /identity/v3/role_assignments?user.id=826539d8549b40ddacbe997a519691ad&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:20.191331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:20.252035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b85a-abca-4ffd-83e1-49a01d5a1178 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.290234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.290937 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.291225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.291526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.292293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:20.292606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.294034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.306003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 381/757] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:20.311523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:20.437057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961c763-1ebc-4c4d-b182-a1eb5aadf193 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.439470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:20.479783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.480588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.480871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.481146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.493687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.495158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:20.524534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:20.525488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:20.527008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:20.527008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:20.527008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:20.527008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:20.527008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 377/758] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:20.532463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:20.663661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c82365-08cf-4edf-96f1-dbabaf0439ac None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.683722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:20.701827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.702417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.702706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.702977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.703729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:20.704036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.705416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.718554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 382/759] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:20.724340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:20.830304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73226712-ba97-4b84-bcb0-736976964898 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:20.867196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:20.868105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:20.868387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:20.868660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:20.876513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:20.877945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:20.892554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:20.893515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:20.895442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 378/760] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:20.900402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.054229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c024-8814-4639-a86d-d54b9895eb18 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:21.101696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:21.103178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.103704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.104191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.105220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:21.105723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:21.111375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:21.130126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 383/761] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:21.135341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.161481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:21.264016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb7c32c-81f6-4ba9-9344-dbe58b745562 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:21.300797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:21.301934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.302227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.302499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.318024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:21.319584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-01a56217-f6f6-42ad-8052-19df48391278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:21.342173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:21.343122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:21.344098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:21.344098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:21.344098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:21.344098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:21.344098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 379/762] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:21.355665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.391100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:21.489138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbef962-12c0-4941-8b60-bfe8746056c6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:21.537216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:21.537938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.538216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.538533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.539254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:21.539599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:21.540964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:21.548921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf012-ff34-42d7-b04b-72b8088b11e2 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.549208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf012-ff34-42d7-b04b-72b8088b11e2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.549489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf012-ff34-42d7-b04b-72b8088b11e2 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.552373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 380/763] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:21.556139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 384/764] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:21.559399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5e26-9a10-4f6b-a85c-3c996b0b0c70 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.559399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5e26-9a10-4f6b-a85c-3c996b0b0c70 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.559399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5e26-9a10-4f6b-a85c-3c996b0b0c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.562223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.598409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-9c6a5e26-9a10-4f6b-a85c-3c996b0b0c70 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:21.600568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9c6a5e26-9a10-4f6b-a85c-3c996b0b0c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:21.704907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:21.716625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc924c-94b5-465b-b50e-fe6bc12cbb87 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:21.740205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:21.748100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 381/765] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:21.755934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:21.756476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aa2b8-e298-468c-9757-39bb686d41e7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.756765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aa2b8-e298-468c-9757-39bb686d41e7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.757037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771aa2b8-e298-468c-9757-39bb686d41e7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.757657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.758141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.758611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.758785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 382/766] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:21.763717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.763999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.764274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.767522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:21.769383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:21.785234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:21.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:21.787453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:21.790077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 385/767] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:21.790413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:21.795496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:21.795794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.865643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 383/768] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:21.872802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.906290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721c53-732b-4d09-9547-03b2d58ccbd3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:21.944596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:21.945860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:21.946367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:21.946863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:21.947891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:21.949583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:21.965298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:21.966255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:21.967103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:21.970121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 386/769] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:21.971100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:21.989493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:21.992738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3372eed-9382-4006-b90c-b4e01f07429a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.022873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.032063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.032835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.033181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.033456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.034304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.034685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.036205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.054082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 384/770] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:22.054082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.090893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b9e6d-5edb-4f15-8562-074c1b092c31 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.128515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.130793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.131370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.131852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.143009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:22.153075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:22.154057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:22.154919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:22.157907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 387/771] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:22.158889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.167643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e85995-9898-41b4-93c1-82021d34326b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.206766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.208239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.208449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.208729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.209504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.209819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.211251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.216880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 385/772] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:22.217768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.243983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.272454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de9e02-afd0-4a93-9dc7-f92b6020faaa None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.278328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.311266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.312649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.313137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.313632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.316226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=0364cf432f5b44a9a62f08c4243fff5e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.318508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.333148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc42fb-e279-472f-bc45-6743115dd5ec None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.341781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 388/773] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:22 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:22.341781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.407328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.407941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.408224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.408498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.409269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.409578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.411798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.436653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 386/774] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:22.451107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.549653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a75f9a-8df7-450a-8894-39bdc95a17fb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.590908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.591616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.591902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.592181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.593005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.593317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.594698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.596033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.605774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 389/775] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:22.611119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.617143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f64f8-e756-4b96-ab0c-5f7b8cb12a19 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.636463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.673716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.675000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.676789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.678604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.697491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.699309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-bcec2dd1-8023-4757-a671-eecb0c3903fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:22.718700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:22.719943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:22.721312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:22.721312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:22.721312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:22.721312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:22.721312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 387/776] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:22.721729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aef56-00d3-48a7-9b1a-ac331fd9dcd9 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.721923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aef56-00d3-48a7-9b1a-ac331fd9dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.722206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3aef56-00d3-48a7-9b1a-ac331fd9dcd9 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.728713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 388/777] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:22.728713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aef56-00d3-48a7-9b1a-ac331fd9dcd9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.756797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb6896-a43f-4c15-9734-1f0e995edb20 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.790929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.791632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.792015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.792303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.793100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] RBAC: Adding query filter params (interface=public, service_id=fb0aa3ab17714e41b777cf30df0e2da6, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.793415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.794790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001df0d7-e5ec-47ab-b012-8a058509fec6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.803000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 390/778] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/endpoints?service_id=fb0aa3ab17714e41b777cf30df0e2da6&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:22.804298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf86f5b-6717-4286-a296-c157f1af44b0 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.804585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf86f5b-6717-4286-a296-c157f1af44b0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.804860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf86f5b-6717-4286-a296-c157f1af44b0 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.810135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 391/779] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:22.810135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109b62-6919-4227-b136-4050d6fe6101 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.810135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109b62-6919-4227-b136-4050d6fe6101 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.810135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81109b62-6919-4227-b136-4050d6fe6101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.825394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-81109b62-6919-4227-b136-4050d6fe6101 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:22.826913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-81109b62-6919-4227-b136-4050d6fe6101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:22.847278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3aef56-00d3-48a7-9b1a-ac331fd9dcd9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:22.883449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:22.884120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.884402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.884683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.885400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:22.885704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:22.887064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:22.888814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:22.899487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 389/780] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:22.902050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 392/781] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:22.904616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:22.905517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:22.906016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:22.906497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:22.930053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:22.931511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:22.939827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.006856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 393/782] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:23.008375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.019130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926014f-3c9c-40bf-92fb-24f2b5d82b0f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.058956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.059679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.059962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.060237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.060912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.062278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.068189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 390/783] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:22 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:23.068973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.142836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aad07-f3e4-4d87-aea1-7d1f16d7c8d3 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.174311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.190394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.191854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.192458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.192991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.201527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.203456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.205682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.218797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceec7bc-5754-4405-b5cf-4441e8bd73d7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:23.226419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:23.227491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:23.228434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:23.233308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 394/784] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:23.234263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.267896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.268693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.269163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.269292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] PATH_INFO: `/v3/services/network` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.270407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.271819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:23.286781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:23.288106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:23.288973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 391/785] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:23.289813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.362595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15cdc1-cbc8-45b4-a48c-0d03770f1751 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.399777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.400572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.400860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.401133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.412313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:23.423126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:23.424260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:23.425161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 395/786] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:23.427065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.429150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ee7c7-76b4-47f4-b546-65d0b2118876 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.476245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.477282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.477792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.478270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.479948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:23.480527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.482659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.483566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.508645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 392/787] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:23.508645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.525591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.573343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f5e8-4aac-4f86-9474-c794ac01af57 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.616512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.617198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.617670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.617948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.619085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] RBAC: Adding query filter params (name=network) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:23.619460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.621139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.628087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 396/788] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:23.628995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.629938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cac576-4837-4e6c-9342-8bcd79281f29 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.674241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.675166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.675617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.676642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.678154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:23.678629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.680888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.691853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 393/789] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:23.692887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.758387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672d926-d325-43b3-b0cd-bce4f59adddf None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.767798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.788558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.810866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.812477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.813019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.813475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.820718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c4fc4-cf42-47d7-9ff4-69a2b24c5e81 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.831192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.833911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.857765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:23.858559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:23.858936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:23.859221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:23.860302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:23.861417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:23.861417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:23.861417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:23.861417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:23.861704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:23.862307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] RBAC: Adding query filter params (type=network) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:23.862702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:23.868300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:23.868449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 397/790] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:23.868449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.874915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 394/791] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:23.876091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:23.994298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa162b-37c7-4264-91b4-2add2ad24bed None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:23.994504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5433a3a-8def-4c51-a184-ca0e48d5644d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.033744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.034486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.034737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.034996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.037524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.038798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.039308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.039791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.040711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:24.041238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.043361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.046220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.047731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.053946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.066641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 395/792] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:24.067004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:24.067930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:24.068845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:24.068845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:24.068845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:24.068845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:24.070350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.072868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 398/793] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:24.073793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.086771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.181941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffcd53-9a25-4aa4-a8fc-6b5d6f4a5bdb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.208836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366fa8b-3f00-422d-a1d6-c18054e76f49 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.235047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.237393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.238150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.247074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.248999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.250775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.251587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.251870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.252146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.253052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] RBAC: Adding query filter params (interface=public, service_id=85153d3d9c2642289e3e0c1132c28a7c, region_id=RegionOne) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:24.253368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.255657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:24.266904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:24.267863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:24.268773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:24.269214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 396/794] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/endpoints?service_id=85153d3d9c2642289e3e0c1132c28a7c&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:24.270095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.271818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 399/795] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:24.276586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.353652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.361736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.387612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43eea2-da86-4f40-b658-a9b6674c222e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.406857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb871f0-b422-4cfa-b66d-c60557582968 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.425139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.425838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.426201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.426480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.427327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.428642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.443187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 400/796] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/services => generated 772 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:24.445051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.450432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.451879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.453582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.453795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.454675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:24.454994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.459336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.484689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 397/797] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:24.496474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.628162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da5bf4-1c72-46d3-a48c-cfdea2f30fb3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.645552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce80dfb-8df4-421b-9e22-68c85849e354 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.668732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.669831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.670115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.670932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.671814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:24.672282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.674387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbfe43-b143-4ebd-a255-2bc8f25a23a7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.677503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.683930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:24.685552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.686453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.686739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.687603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.693388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 401/798] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:24.695358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15356ec3-c0f7-458a-927c-614603244b2c None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.695646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.696067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15356ec3-c0f7-458a-927c-614603244b2c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.696575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15356ec3-c0f7-458a-927c-614603244b2c None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.697079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.698500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 402/799] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:24.704071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15356ec3-c0f7-458a-927c-614603244b2c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-8a4996c8-57a0-4631-a797-b942ae4b4e95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:24.718494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:24.719520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:24.720501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 398/800] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:24.721585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.721585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.721738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.744362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:24.745807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:24.829707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15356ec3-c0f7-458a-927c-614603244b2c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.842415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 399/801] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:24.843630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.893510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:24.895782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:24.896775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:24.897546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/1ab2924b8a9d41a7a6b74a73a7b809bd/roles/09104180e3694124bf32296605caf033` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:24.916138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=1ab2924b8a9d41a7a6b74a73a7b809bd, role_id=09104180e3694124bf32296605caf033)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:24.917938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:24.937703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 403/802] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:24 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/1ab2924b8a9d41a7a6b74a73a7b809bd/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:24.938670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:24.974033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1283b6-3764-49e4-a99f-268b8f9d7098 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:24.990800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.005091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.044617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.045118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.045660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.045934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.046785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:25.047093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.048585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.094914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 400/803] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:25.095969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.100235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924232d-4a28-429f-9e47-d87682271b06 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.146569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.147300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.147589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.147857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.160883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:25.174446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:25.175471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:25.177029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 404/804] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:25.182676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.237751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede80be9-c040-4d67-b0a5-518560ca2f49 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.284012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.284830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.285120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.285955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.292317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf990f45-3783-4cf7-904a-03fe43aa0370 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.295875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.308534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.310001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.328945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.329616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.329895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.330670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.331452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:25.332782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.334181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:25.343821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:25.344722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:25.345641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:25.345641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:25.345641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:25.345641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:25.345641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 401/805] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:25.351312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.353888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 405/806] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:25.362444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.388796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.480468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dee2b2-0d9b-4a70-b5ef-8d976250cd02 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.498806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5323-5d96-40e8-a109-0e52fe9b0b08 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.529366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.531142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.531762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.532353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.534492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.535149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.535516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.535790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.536768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:25.536768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.538188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5c98-6ca3-47a4-bedf-fe534fda2db8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.548031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.550022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.552243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 402/807] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:25.553564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541c65-213c-4769-86ca-8984a6932f8c None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.553848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541c65-213c-4769-86ca-8984a6932f8c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.554118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35541c65-213c-4769-86ca-8984a6932f8c None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.559489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 403/808] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:25.559489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541c65-213c-4769-86ca-8984a6932f8c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-e7309246-216c-4c61-93aa-208012b69c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:25.580147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:25.581139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:25.582762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:25.582762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:25.582762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:25.582762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:25.589572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 406/809] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:25.589572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.589572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.589572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.609108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:25.610324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:25.631000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.678033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.711677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35541c65-213c-4769-86ca-8984a6932f8c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.711861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 407/810] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:25.712991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.752629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.753684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.754486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.754821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.764630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.767255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:25.786641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:25.787867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:25.788774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:25.791371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 404/811] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:25.792438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.842036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23faddb-dfc2-48a4-b19b-2341276e3e6b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.886570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.887631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.888119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.888592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.889538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:25.890045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.891644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.907634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 408/812] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:25.908778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.916703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36c3d3-86c1-4a08-b359-882ebf4a81f0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:25.923236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:25.964885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:25.965628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:25.965908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:25.966191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:25.967235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:25.968624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:25.986214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:25.987180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:25.988071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 405/813] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:25.989232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:25.998799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.103508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c845f-d5a6-465e-bef2-4b5dfaf1e4a8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.143745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.145464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.145857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.146570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.156613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.158892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.161567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b165ae-825b-4f9b-8f65-5fe5d975c628 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:26.184699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:26.185735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:26.186710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:26.191489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 406/814] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:26.193262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.208310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.209780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.210289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.210733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.211918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:26.212407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.214080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.227282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 409/815] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:26.228438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.276344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.320823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae52db1-083e-4d07-955d-afacd9f0b89b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.332910 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.368875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.369918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.370098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd2df-2c25-4f5f-9d5c-1f135066b712 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.370600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.371085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.372174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:26.373440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.375040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.382361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 407/816] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:26.383431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.409758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.410534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.410814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.411142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.412036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:26.412352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.413782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.448576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 410/817] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:26.449648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.517484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a16b-a864-4615-8e0d-1d23246b2084 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.556145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324bd6d-47fc-43c9-8da3-47abd7d32b30 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.557208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.565059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.567107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.567718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.568189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/826539d8549b40ddacbe997a519691ad/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.571511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=826539d8549b40ddacbe997a519691ad, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.573343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.587689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.595368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 408/818] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:26 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/826539d8549b40ddacbe997a519691ad/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:26.596508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.609482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.610465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.610953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.611426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.612462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:26.612777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.614497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d29d8-1f86-4211-b5d0-e9822e68b409 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.621685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 411/819] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Jul 20 09:23:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:26.623527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c210-9960-41cf-b572-054446a1744d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.624038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c210-9960-41cf-b572-054446a1744d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.624526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c210-9960-41cf-b572-054446a1744d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.631077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 412/820] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:26.632357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3c210-9960-41cf-b572-054446a1744d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:26.751596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5bb4a-844b-46a0-93fc-cf975769b513 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.758327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3c210-9960-41cf-b572-054446a1744d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:26.797459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.798040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.798321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.798594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.804085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:26.804744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.805028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.805304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.806047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.807631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40a315-66b4-42fc-883d-0723e4540959 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.834050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 413/821] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:26.835162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:26.836330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:26.836620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:26.864163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:26.868326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:26.892513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:26.915897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:26.917398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:26.958231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.008169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 414/822] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:27.034409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.089434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 409/823] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:23:27.098112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.178116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fe212-90b9-4862-a348-0703360c0122 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.203640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.215545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0549b74-365f-46bc-9bf3-aee3871f246d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.229115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.230802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.231229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.231918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] PATH_INFO: `/v3/services/network` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.233200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.234988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-e2599362-9a8a-4a7f-8d4f-bce08f8eaacc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:27.253288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:27.254962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:23:27.255856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:27.256605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 415/824] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:27.258660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc2b86-58cd-429f-b458-cafc0b0f0732 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.258944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc2b86-58cd-429f-b458-cafc0b0f0732 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.259214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc2b86-58cd-429f-b458-cafc0b0f0732 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.260735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 416/825] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:27.262501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.263334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.263615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.263892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.264880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc2b86-58cd-429f-b458-cafc0b0f0732 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.277460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.278913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-f3326f4b-e938-4d9d-90d3-54e62e330f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:27.298478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:27.299466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:27.300398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:27.300398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:27.300398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:27.300398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:27.300398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 410/826] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:27.301946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.302236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.302510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.309724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.322008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:27.323328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:27.384674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc2b86-58cd-429f-b458-cafc0b0f0732 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.397955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 411/827] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:27.398814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.423647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.424917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.425428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.426316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.427303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] RBAC: Adding query filter params (name=network) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:27.427818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.429692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.435440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 417/828] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:27.441338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.493748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.524496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6526-a5f1-44e9-a09f-8614327d7567 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.555131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.558764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f37d4-4552-4c6f-a608-e1999ca1093d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.560726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.561409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.562427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.562427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.563125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:27.563460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.564930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.578235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 412/829] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:27.579072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.599214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.600415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.600902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.601376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.602356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] RBAC: Adding query filter params (type=network) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:27.602962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.604616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.615489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 418/830] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:27.616844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.685969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef285a92-47c8-49ef-b945-9e38639a8b30 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.716311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.722369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.724135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.724515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.724880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.730344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40694d-ed2b-4812-83c5-47de8ac98370 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.736184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:27.744642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:27.745767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:23:27.746679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:27.751028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 413/831] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:27.751961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.772314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.773499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.773988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.774461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.782008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.783855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:27.799625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:27.800611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:27.802116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:27.804782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 419/832] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:27.805856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.833637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.865547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f020fbc-3948-4532-bfe5-1ffa37c0c4be None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.902724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.903653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.904074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.904629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.905610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.907282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:27.917162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d82ea-4035-46c9-85d8-15b6add27a61 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:27.930625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 414/833] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:27.931876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:27.987793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:27.992453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:27.993240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:27.993598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:27.993947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:27.995817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:27.995943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:27.997465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.014382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 420/834] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:28.021127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.055767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cc733-c3cf-40c2-8845-bc4330c1e5fd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.100797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.101780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.102236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.102683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.103860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:28.104343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.105995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.137313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.146132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 415/835] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:28.152000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.169907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46e25-c72e-4a76-baa2-b6e7a8ebd307 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.237394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.238250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.238566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.238924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.251308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.252874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:28.274478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:28.275516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:28.276429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:28.276429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:28.276429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:28.276429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:28.279527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 421/836] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:28.284490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.323105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1743d-0945-4e45-a010-35d5032fc876 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.338277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.374984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.376304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.377371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.377888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.379366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:28.379898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.382650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.391167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 416/837] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:28.427739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0bb55-ddd9-422f-93d8-f9acfcfdc8df None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.479966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.480646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.480929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.481199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.481931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:28.482235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.483650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.485427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.499202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 422/838] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:28.504944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.613042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727ab4f-9c6b-4c8e-9c95-ea83fbd8a604 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.648460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.649276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.649553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.649823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.657393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.658892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-1e84ad4f-e02d-4956-bcf4-70ed9cbf829b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:28.674897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:28.676540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:28.677687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 417/839] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:28.682839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.785998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972220d-1f8c-4557-8dba-a2931c7d45ef None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:28.821934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:28.822607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.822887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.823161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.824027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:28.824334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:28.825762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:28.834981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb319-f7c5-4b5c-9e65-607b7ec93e48 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.835271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb319-f7c5-4b5c-9e65-607b7ec93e48 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.835687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fb319-f7c5-4b5c-9e65-607b7ec93e48 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.837310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 418/840] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:28.842174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.842460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.842737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.849070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 423/841] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:28.854432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:28.862891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:28.865196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:28.898915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.913299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.968362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41877600-fe88-4b17-9ea7-a2b5004afa9c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.006926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.007803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.008183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.008556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.071431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 419/842] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:29.086138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.087906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.095942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.149743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.212896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 424/843] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:23:29.218293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.222686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09155c29-7c15-4c33-9779-228541abb0fc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.258762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.259951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.260437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.260940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.273822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-f5e91929-ea9e-44a0-b18d-a50c7f32b4f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:29.281411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:29.282357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:29.283227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 420/844] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:29.284887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef012b-f0ed-49c9-8f24-c65f3de3f331 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.284887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef012b-f0ed-49c9-8f24-c65f3de3f331 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.284887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef012b-f0ed-49c9-8f24-c65f3de3f331 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.287338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 421/845] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:29.290802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef012b-f0ed-49c9-8f24-c65f3de3f331 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.317566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.336624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a318598-c2c3-4fa2-a2eb-7cfd6489c60f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.372670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.373866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.374148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.374419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.385951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.387425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.400348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef012b-f0ed-49c9-8f24-c65f3de3f331 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5bb43f76-c14d-4054-a570-83af003bc487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:29.404683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:29.405587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:29.406527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:29.406527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:29.406527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:29.406527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:29.406527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 425/846] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:29.407395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.407681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.407955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.414000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.425101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:29.426786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:29.451183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.452100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.452527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.452945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.453829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:29.454280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.455835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.465536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 422/847] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:29.466608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.499037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 426/848] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:29.500185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.561502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.579591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f115-9818-43dd-98f4-dce09baf5e11 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.618397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.619043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.619433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.619709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.620302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:29.620607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.621543 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b355fc-4821-47f2-8097-0d92ee0477fc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.622000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.635052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 423/849] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:29.640244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.658741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.659565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.659842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.660118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.662761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.673323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.674762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:29.695121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:29.696109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:29.697008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:29.697008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:29.697008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:29.697008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:29.698064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 427/850] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:29.698335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.757293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23dd7e-fde5-4cbd-a16c-2fcb0c8254e6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.791566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.792362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.792649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.792922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.793569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.794923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:29.807619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:29.808622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:29.809570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 424/851] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:29.810399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.812720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a401960-0bef-4431-bfdc-b018995f2bab None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.822225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.853395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.853933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.854213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.854486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.861223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.862647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:29.876563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:29.877460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:29.878376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 428/852] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:29.878880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.921240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d152e67-6d4e-4683-8f4d-f4b7f67094e3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.928939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:29.955979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.956649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.956932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.957752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.958650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:29.958957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.960356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.972542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 425/853] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:29.974156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.989207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984fafc-9b47-4b3e-8669-60fc267248fd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.025926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.026583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.026860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.027125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.027862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:30.028163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.029508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.039208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 429/854] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:30.040074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.045948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.083258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d58f4-8e59-4da2-95ba-65746d604e12 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.118360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.119020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.119317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.119731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.120979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:30.121298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.122711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.135911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 426/855] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:30.136999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.162094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac2f6c-4cc7-4d31-afc1-fbea24766b8b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.175626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.203639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.205087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.205592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.206113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.218218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.219689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:30.235331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:30.236490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:23:30.237587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:30.240227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 430/856] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:30.241159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.251925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b88ce-3fe6-47a4-9572-2c39b89f8fc4 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.283004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.294602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.295372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.296364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.296652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.297639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:30.297951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.299838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.315320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 427/857] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:30.315320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.356447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef86f76-9ae4-4085-a48c-7fbbb8a860e9 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.398287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.398967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.399342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.399696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.400611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=1ab2924b8a9d41a7a6b74a73a7b809bd) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:30.401014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.402471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.409513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 431/858] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/role_assignments?user.id=1ab2924b8a9d41a7a6b74a73a7b809bd&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:30.456909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e43b7-f857-4ba8-9e5b-e49f838e37b7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.496734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.497418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.497695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.497963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.498807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:30.499112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.500586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75785c-717c-4ea1-8f3a-44d39e893af5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.523401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 428/859] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:30.527873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.569792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.585699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fb3c5-7558-4c2b-babe-50b832b441bb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.585699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fb3c5-7558-4c2b-babe-50b832b441bb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.586002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fb3c5-7558-4c2b-babe-50b832b441bb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.587530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 429/860] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:30.592756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.593042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.593322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.610859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:30.612208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:30.645297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974226aa-e89d-49bc-bb4b-b89a5963d799 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.667019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.695736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.696400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.696682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.696955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.700247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 430/861] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:30.724809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.783830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.786425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.830768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:30.840881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2213970-17d9-465f-835c-d2e7eb4f81a1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.882296 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.883526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.883959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.884962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.885777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.887801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:30.905915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:30.907117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:30.908211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:30.916387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 431/862] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:30.917682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.925703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 432/863] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:23:30.927769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.977581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:31.041701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a08d7-eb2b-48cb-a5a9-0177d4f31b6d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.051818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de75e1-8972-474f-ab03-718a08b992c7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.088819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.089588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.089875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.090158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.091008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:31.091276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.092491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.092856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.094395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.095321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.095807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.103232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 432/864] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:31.103911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:31.104258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.116152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.117618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:31.136540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:31.137525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:31.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:31.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:31.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:31.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:31.138450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 433/865] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:31.143708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.240846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d2bfd-ed6f-41d5-8009-9a0a9ab5283f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.252477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:31.281353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754dfef-0b5b-49da-9793-a97965d3c187 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.283280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.284186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.284641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.285082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.285990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:31.286469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.288042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.299735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 433/866] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:31.321271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.322017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.322355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.322661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.323586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:31.323954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.325359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.344694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 434/867] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:31.346356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.458939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ab53-de99-4d51-8ea1-3eeb4a936792 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.494870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.495623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.495962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.496234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.504997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.506481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:31.521931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:31.522910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:31.523830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:31.527045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 434/868] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:31.529871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.581354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:31.638159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c90a86-a64f-49d3-bb87-1e22bb78a9cf None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.673821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.674490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.674768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.675039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.675889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:31.676195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.677694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093fb34-ccc5-4f67-9b4b-40301272c554 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.695415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 435/869] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:31.697441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.734788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:31.807535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534d871-8a6a-4f19-9e23-2d8abaa92017 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.841804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.842462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.842746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.843025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.843881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=826539d8549b40ddacbe997a519691ad) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:31.844194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.845720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd9c4c-72e8-4f16-9ab5-fcd27a32bd95 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.852653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 435/870] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/role_assignments?user.id=826539d8549b40ddacbe997a519691ad&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:32.499096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fa4d4-bbfd-4262-a60a-e4b08e527d1b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.499767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fa4d4-bbfd-4262-a60a-e4b08e527d1b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.500267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fa4d4-bbfd-4262-a60a-e4b08e527d1b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.502143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 436/871] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:32.507573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2557d2-1852-40ae-bfc8-ebeba556c187 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.508098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2557d2-1852-40ae-bfc8-ebeba556c187 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.508656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2557d2-1852-40ae-bfc8-ebeba556c187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.525864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-bd2557d2-1852-40ae-bfc8-ebeba556c187 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:32.527413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bd2557d2-1852-40ae-bfc8-ebeba556c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:32.597566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 436/872] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:32.621009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fa4d4-bbfd-4262-a60a-e4b08e527d1b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:32.668342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd95f2-3102-47d0-a578-7f7f8984fa31 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.668807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd95f2-3102-47d0-a578-7f7f8984fa31 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.669091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd95f2-3102-47d0-a578-7f7f8984fa31 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.671078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 437/873] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:32.676117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.676407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.676687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.696242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:32.697609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:32.723656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:32.742513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fa4d4-bbfd-4262-a60a-e4b08e527d1b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:32.751757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:32.782303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:32.787452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 438/874] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:32.787573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.787573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.787573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.787573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:32.787573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:32.803594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 437/875] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:32 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:32.812513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:32.928367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fab78e-88fb-4197-95b3-3a87186ea19d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:32.977156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:32.977892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:32.978173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:32.978441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:32.990315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:32.998571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:32.999585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:33.000437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 439/876] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:23:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:33.006400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.117741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488a919-c73e-4858-b230-45e8b5a71eb1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.152755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.153471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.153753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.154025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.154765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:33.155068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.156467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.165095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 438/877] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:33.172373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.188156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:33.276854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592874-de5c-4663-abdd-7f0297e00da0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.314810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.315638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.315912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.316186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.328539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.329997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-86a2a79b-b304-431e-bbab-b85a6ea9119a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:33.348559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:33.349487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:33.350405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:33.350405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:33.350405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:33.350405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:33.350405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 440/878] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:33.356455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.386981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:33.436287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4afb-ee0e-4feb-a1c5-5beff7b181e9 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.436721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4afb-ee0e-4feb-a1c5-5beff7b181e9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.437119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e4afb-ee0e-4feb-a1c5-5beff7b181e9 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.438865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 441/879] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:33.444984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc149-3377-412b-85ca-219924af0c02 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.445393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc149-3377-412b-85ca-219924af0c02 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.445770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc149-3377-412b-85ca-219924af0c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.468219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-914fc149-3377-412b-85ca-219924af0c02 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:33.470215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-914fc149-3377-412b-85ca-219924af0c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:33.475924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677e090-1291-45dc-a7ae-ff36fd29d136 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.511445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.512107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.512388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.512704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.513376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:33.513785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.515170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.528494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 439/880] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:33.533735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.546385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:33.563738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 442/881] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:33.578332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc149-3377-412b-85ca-219924af0c02 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.646106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:33.664397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b56b2a-24bc-4550-83fe-a30c084d096d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.685759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc149-3377-412b-85ca-219924af0c02 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.705282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.706346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.706802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.707989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.715931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.717676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.722505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.723776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.724061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.724330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.725063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.726417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:33.736043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:33.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:33.738645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:33.741055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 440/882] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:33.746673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 443/883] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:33 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:33.748234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:33.862872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fd441-9514-496f-8cee-9f13980a0414 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:33.912361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:33.913447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:33.913950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:33.914445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:33.915531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:33.916051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:33.917733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7a5ea-c30c-4c1c-b449-d2a00342f9d6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:33.937864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 441/884] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:33.943729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.006814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.044604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b7c9c-20ca-4842-b5f1-a6aaf85d27b4 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.044922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b7c9c-20ca-4842-b5f1-a6aaf85d27b4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.045149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b7c9c-20ca-4842-b5f1-a6aaf85d27b4 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.047946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 442/885] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:34.054570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.054852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.055128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.058564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad75c-e1e0-4fda-9bb6-4b7e3dce7bda None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.060391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.073784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:34.075221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:34.105515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.106435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.106718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.107459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.174011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 443/886] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:34.186195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.187718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.196872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.247364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.311561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 444/887] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:23:34.318870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.324408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4c95c-43ad-48b2-bfe3-1a2b9593f8db None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.355812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.361915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.362994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.363533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.364012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.364864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.366450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:34.380791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:34.382035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:34.383082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:34.387550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 444/888] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:34.389560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.432626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a519925-9311-4e59-bfa0-52963e4f9ce0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.476655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.477842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.478325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.478795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.489611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.491369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.503068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11ceaf-fe55-4ab0-a216-5555586838a1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:34.514475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:34.515800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:34.516721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:34.516721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:34.516721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:34.516721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:34.516721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 445/889] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:34.525049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.538932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.546141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.546817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.547200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.547588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.548309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:34.548641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.550147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.556165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 445/890] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:34.561325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.640462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.664602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43289-4dc4-49ec-9e3a-1e408edfa47f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.698039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcc24a-75ce-4cde-b0e0-1a993b92e35e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.715111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.716591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.717159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.717729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.718952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:34.719769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.721283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.738292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 446/891] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:34.747551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.747551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.747551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.747551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.747916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:34.747916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.747916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.748104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:34.759316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 446/892] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:34.836088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:34.878492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bf137-2a69-47bf-bc90-ec80164fa660 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:34.918291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:34.919106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:34.919394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:34.919668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:34.929809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:34.931263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-80b54776-b026-4a1d-97aa-60e509d7913a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:34.948343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:34.949261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:34.950158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 447/893] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:34.956763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.054772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1bd7-bb50-4f3d-9932-fbbe12b5cdaa None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.055051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1bd7-bb50-4f3d-9932-fbbe12b5cdaa None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.055626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d1bd7-bb50-4f3d-9932-fbbe12b5cdaa None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.056960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 448/894] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:35.061895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.062174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.062450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.074880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a74b8-3e1c-4380-b534-bf193d37b1c2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.079933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:35.081105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:35.121391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.122507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.124463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.124901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:35.125357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.126697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:35.127218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.130753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.153991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 447/895] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:35.155051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.163007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 449/896] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:35.187082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.248067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:35.264932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838c69a-1a66-42c3-9695-6f8e97f351bd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.297330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2795d-5fbc-4dfc-b7b7-8cfac3adfc7b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.300210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.301576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.302177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.302276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.303166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] RBAC: Adding query filter params (role.id=c476d036502841398d096ac28f36b62e, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=1ab2924b8a9d41a7a6b74a73a7b809bd) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:35.303510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.304956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.311321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 448/897] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/role_assignments?user.id=1ab2924b8a9d41a7a6b74a73a7b809bd&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=c476d036502841398d096ac28f36b62e => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:35.349182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.350431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.350953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.351511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.352531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.354140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:35.368566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:35.369660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:35.370534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:35.373741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 450/898] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:35.379433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.435888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:35.502576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472344f-6b76-42ce-afd3-4130bff0bcf6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.540544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.541867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.542002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.542159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.543914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:35.543914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.545647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.550476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 449/899] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:35.556686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.659241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2e217-9b52-4c4b-8152-fadf8bd43e82 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.681310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:35.696301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.696968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.697250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.697526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.698230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:35.698534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.699926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.705197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 451/900] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:35.712503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.822530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665021e7-7342-4007-a787-57b8ad223642 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.836432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:35.860015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:35.860677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.860959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.861230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.862021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] RBAC: Adding query filter params (interface=public, service_id=c7d0221d018c4e38a4e78cc7384cb594, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:35.862329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:35.864267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b9ad-21ae-412d-9bd6-a030dc8405a1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:35.869985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 450/901] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/endpoints?service_id=c7d0221d018c4e38a4e78cc7384cb594&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:35.876406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:35.987627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55b0aa-491d-4bf4-8250-d539d5261606 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:35.995156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f8b6-cc39-4d97-af31-aca7b126c4cf None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:35.995466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f8b6-cc39-4d97-af31-aca7b126c4cf None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:35.995792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f8b6-cc39-4d97-af31-aca7b126c4cf None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:35.997317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 451/902] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:36.002018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.002303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.002578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.021355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:36.022544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:36.029029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.029982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.030475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.030965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.031923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.033498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.040360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 452/903] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:35 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:36.103757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 452/904] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:36.128206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:36.185473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:36.246672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a7db7-1876-491e-bff5-cee694155062 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:36.288291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.289113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.289387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.289654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.290304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.291675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:36.305224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:36.306254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:36.307583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 453/905] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jul 20 09:23:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:36.312584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:36.423049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11583d5a-b92a-4daa-8780-7f869ce20c06 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:36.434554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:36.462415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.463083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.463384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.463658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.464360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:36.464661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.466023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.471198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 453/906] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:36.478230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:36.599324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bdea3-81a0-4f62-809a-7bd5c9e1fa6d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:36.612832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:36.635571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.636246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.636528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.636801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.637505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:36.637809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.639163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.644490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 454/907] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:36.651150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:36.757979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce5f8-d5cd-4442-babf-5e7ccc1560c5 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:36.794344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.794999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.795276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.795572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.796674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] RBAC: Adding query filter params (interface=public, service_id=d91d8d6a106443d6a28c541fd85f1bb2, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:36.796990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.798342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1308a77-b2eb-4299-865c-8b67a810ba08 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.805180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 454/908] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:36 2022] GET /identity/v3/endpoints?service_id=d91d8d6a106443d6a28c541fd85f1bb2&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:36.810648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:36.919144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339ea45-adb7-4d35-a538-6abade33b989 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:36.950030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c069c-bf43-41f9-bacc-4912c0b00ac4 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.950311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c069c-bf43-41f9-bacc-4912c0b00ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.950722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c069c-bf43-41f9-bacc-4912c0b00ac4 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.953374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 455/909] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:36.956683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:36.957801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85466e2b-6261-4c39-8d64-04369a74bc24 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.958137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85466e2b-6261-4c39-8d64-04369a74bc24 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.958490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85466e2b-6261-4c39-8d64-04369a74bc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.958604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:36.959058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:36.959557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:36.960487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:36.962475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:36.970939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 455/910] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:36 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:36.979714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-85466e2b-6261-4c39-8d64-04369a74bc24 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:36.981091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-85466e2b-6261-4c39-8d64-04369a74bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:37.012946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:37.063478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 456/911] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:37.081697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:37.136813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:37.202712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a89471-d236-4cb5-85ce-bfcfc28a6ddf None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:37.243194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:37.243904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:37.244184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:37.244456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:37.245190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:37.246555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2533-1b10-4863-9046-dfe9801b1ad1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:37.262333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 456/912] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:37 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:37.562601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b086c-fb8b-45ec-8187-d16ef21b625e None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:37.562995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b086c-fb8b-45ec-8187-d16ef21b625e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:37.563197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b086c-fb8b-45ec-8187-d16ef21b625e None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:37.564886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 457/913] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:37.571031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:37.571394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:37.571683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:37.588843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:37.590032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:37.671416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 457/914] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:37.689930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b086c-fb8b-45ec-8187-d16ef21b625e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:37.798616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b086c-fb8b-45ec-8187-d16ef21b625e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:37.835575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:37.836396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:37.836676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:37.836948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:37.847972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:37.856329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:37.857302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:37.858188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:37.862549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 458/915] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:23:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:37.864112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:37.896392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:37.972522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563f137-8d62-474c-9ffe-523996f5aaba None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.007278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.007980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.008261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.008533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.009281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:38.009586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.010949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.019707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 458/916] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:23:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:38.025685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.052230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.128685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bd6-a3d6-4540-a127-7d95e4fe3de0 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.164130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.164931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.165211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.165481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.176632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.178143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-0439e280-9c5f-4372-ad55-8ab9095b4dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:38.195798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:38.197889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:38.200023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:38.200023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:38.200023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:38.200023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:38.201897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 459/917] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:38.203185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.288015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ccf6-7bab-4516-a0a9-caf686dc0d76 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.288343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ccf6-7bab-4516-a0a9-caf686dc0d76 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.288559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5ccf6-7bab-4516-a0a9-caf686dc0d76 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.290185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 460/918] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:38.298321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.298592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.298901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.299587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.310811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abce1d1-14a5-4e0a-959f-3f247de7207f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.320979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:38.322168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:38.349534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.350376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.350658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.350927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.351685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:38.351987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.353444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.366905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 459/919] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:38.373406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.398717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 461/920] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:38.423254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.431898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.508786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a70d7-1d08-49f1-88ba-f7ac27acd680 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.550227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.551093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e58f9a-8a2e-468e-a328-1a78e87a52a9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.551920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.552417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.553026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.562212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.564539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:38.585327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:38.586316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:38.587311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:38.593551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 460/921] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:38.594172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.600318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.601185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.601466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.601739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.602440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.603898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.608055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:38.622348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:38.623437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:38.624298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 462/922] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:38.630153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.709345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10056cb6-75a9-42b1-a880-f03833f27e21 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.726568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.744978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.746089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.746552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.747018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.748082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:38.748642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.750267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.773591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2432b-e09b-4050-82f2-2c955e773b94 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.774241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 461/923] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:38.775394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.817794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.818436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.818658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.818932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.820365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:38.820685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.822058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.829801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 463/924] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:38.844533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.849367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:38.888276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc7022-9720-460d-a709-04e83fad8905 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:38.927119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:38.927993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:38.928269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:38.928539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:38.939796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:38.941227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:38.960354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:38.961317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:38.962229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:38.962229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:38.962229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:38.962229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:38.966710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 462/925] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:38.967624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:38.980100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48b5b9-52a1-40e0-8642-ca7b0abf1959 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.032650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.033313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.033590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.033860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.034562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:39.034865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.036393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.037763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.044521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 464/926] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:39.050341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.103667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d472bd4-2a01-4c49-941f-7beabac6cee6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.128783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.158179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.159113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.159438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.159714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.161527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:39.161839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.163875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.165610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea45ac6-393a-43b4-8b6d-6030e5c22bfb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.183454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 463/927] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:39.185579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.209837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.210670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.210877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.211157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.212135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.213566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7eca1c-1737-4a9f-8c76-8854d23d8f9e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.234642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 465/928] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:23:39.236122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685b31a-6d93-4305-99da-47138db40986 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.236414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685b31a-6d93-4305-99da-47138db40986 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.236738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685b31a-6d93-4305-99da-47138db40986 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.238308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 466/929] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:39.253002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.253308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.253818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.280251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:39.281799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:39.315042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c7d11-e786-4de0-bcf6-0de5177a6dee None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.353361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.360169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.360937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.361237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.361622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.370075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.371684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.380370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 467/930] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:39.391367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:39.392327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:39.393230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:39.394781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 464/931] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:39.396332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.404187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.452066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.515561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb025ee-4d3d-4b78-bd37-f69b44be0207 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.528718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9add5b-2140-436c-9f91-06ce2ddbbcfb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.554999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.555702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.555987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.556436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.557068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:39.557351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.558784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77718bef-cca4-450f-be3c-7c2c91e4b945 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.564232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.565044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.565320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.565986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.566256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.567655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.575325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 468/932] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:39.575325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d126-e29d-4cd0-8b89-0e5af4b2ebf2 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.575325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d126-e29d-4cd0-8b89-0e5af4b2ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.575325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279d126-e29d-4cd0-8b89-0e5af4b2ebf2 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.575325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 469/933] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:39.577750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d126-e29d-4cd0-8b89-0e5af4b2ebf2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-cbad4bff-0124-4d7a-8d56-241a9e3ca82d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:39.585977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:39.586961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:39.588520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 465/934] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:39.589392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.589392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.589392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.608260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:39.609432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:39.633274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.683029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 466/935] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:39.693067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.694068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279d126-e29d-4cd0-8b89-0e5af4b2ebf2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.728184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.731538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.732640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.732922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.733197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/1ab2924b8a9d41a7a6b74a73a7b809bd/roles/c476d036502841398d096ac28f36b62e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.736216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=1ab2924b8a9d41a7a6b74a73a7b809bd, role_id=c476d036502841398d096ac28f36b62e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.737644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.759320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 470/936] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:23:39 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/1ab2924b8a9d41a7a6b74a73a7b809bd/roles/c476d036502841398d096ac28f36b62e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:23:39.759320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.830885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b856df6-758f-44dd-82c7-ebe47e43a7da None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.873302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.873526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff11f46-f5e4-4da6-b55d-d08cc1329a91 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:39.874239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.874632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.874956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.875834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:39.876325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:39.877184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.880949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.889399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 467/937] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:39.894881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:39.914375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:39.915209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:39.915537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:39.916198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:39.916544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:39.918109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:39.935140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:39.936490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:39.937636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 471/938] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:39.944301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:40.024268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:40.039131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ab776-0ac0-459c-be48-9a7e9dc6ac01 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.066938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b5eab-e1b2-4419-a2ce-748c06a0b9df None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.085632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.086302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.086588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.086864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.087632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:40.087942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.089310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.099697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 468/939] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:40.100547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:40.104691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.105452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.105730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.106003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.106702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:40.107005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.108404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.114333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 472/940] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:40.120171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:40.152065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:40.227459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c80c6e-e57b-4527-a3e8-635da40f3bbb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.232458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954d32c-5734-49ed-8a13-96de1c71f7bc None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.262492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.263156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.263569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.263743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.264452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:40.264754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.266118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.275564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 473/941] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:40.277795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:40.280696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.282512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.282987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.283468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.284415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.286582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.323969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 469/942] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:40.393019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:40.405743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9b090-fd0d-4e80-9615-3f33fbd142f2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.458750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.459880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.460388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.460899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.461967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] RBAC: Adding query filter params (interface=public, service_id=23ddb4a819c44f959a630a4d934c0c86, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:40.462496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.464134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e7e6f-4358-4a46-99d0-16a359577509 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.474670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 474/943] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:40 2022] GET /identity/v3/endpoints?service_id=23ddb4a819c44f959a630a4d934c0c86&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:40.477088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:40.497373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:40.591845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e6dfa-ca42-43dc-9796-cd21b488438c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:40.626982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:40.627734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:40.628015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:40.628291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:40.628970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:40.630392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca97ec-962b-4659-9fa9-3da554311407 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:40.637059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 470/944] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:40 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:41.432578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1479-91cf-47b1-82c8-413a07ebc5b5 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:41.433096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1479-91cf-47b1-82c8-413a07ebc5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:41.433200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1479-91cf-47b1-82c8-413a07ebc5b5 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:41.434494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 475/945] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:41.445886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:41.446320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:41.446498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:41.464069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:41.465268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:41.533837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 471/946] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:41.557416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd1479-91cf-47b1-82c8-413a07ebc5b5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:41.667103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd1479-91cf-47b1-82c8-413a07ebc5b5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:41.702625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:41.703206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:41.703517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:41.703790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:41.704498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:41.705919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:41.707839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING keystone.server.flask.common [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 09:23:41.725841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.sql.core [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:23:41.725841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 09:23:41.725841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [parameters: {'id': '960c8a542e7a4457b0088ca39821ceff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 09:23:41.725841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80185) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 960c8a542e7a4457b0088ca39821ceff.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 960c8a542e7a4457b0088ca39821ceff. Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:23:41.733615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result.read() Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 09:23:41.734448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application yield resource Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise self.value Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application session.commit() Jul 20 09:23:41.735451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.session.flush() Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application compat.raise_( Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 09:23:41.736351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 09:23:41.737237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application result.read() Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:23:41.738030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [parameters: {'id': '960c8a542e7a4457b0088ca39821ceff', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:41.740265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:41.742141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 960c8a542e7a4457b0088ca39821ceff. Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:41.744256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 476/947] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Jul 20 09:23:41 2022] POST /identity/v3/projects => generated 262 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:41.753492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:41.760989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:41.860436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e02ba-d99a-43f1-809a-a84b1b4a329b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:41.899919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:41.900716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:41.901004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:41.901281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:41.908060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:41.909498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-24970739-ccaf-45c5-9ddf-75f8cbcc9cef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:41.923854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:41.924769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:41.925692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 472/948] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:41.933852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:41.951048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:42.029772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0eb2ea-c872-4cf9-a20c-10eb7935cdca None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.030090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0eb2ea-c872-4cf9-a20c-10eb7935cdca None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.030336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0eb2ea-c872-4cf9-a20c-10eb7935cdca None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.031996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 473/949] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:42.036958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef083267-1047-462c-89ed-59626f47a60f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.037246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef083267-1047-462c-89ed-59626f47a60f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.037523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef083267-1047-462c-89ed-59626f47a60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.045838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569da229-a739-4fa1-83f4-4237daa68a26 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:42.055054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ef083267-1047-462c-89ed-59626f47a60f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:42.056293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ef083267-1047-462c-89ed-59626f47a60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:42.090735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:42.091434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.091720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.091996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.092757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:42.093070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:42.095004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:42.125714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:42.126844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 477/950] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:42.137865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 474/951] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:42.162234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.244661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:42.278556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b003d7f-e73d-460d-9966-92e3859f76e1 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:42.318886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182f77-539b-419c-8279-711fa450040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:42.319680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182f77-539b-419c-8279-711fa450040f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.319967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182f77-539b-419c-8279-711fa450040f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.320245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43182f77-539b-419c-8279-711fa450040f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.337690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43182f77-539b-419c-8279-711fa450040f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-43182f77-539b-419c-8279-711fa450040f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:42.346054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:42.347048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:42.348563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 478/952] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:42.355603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef083267-1047-462c-89ed-59626f47a60f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.478903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef083267-1047-462c-89ed-59626f47a60f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:42.520212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:42.521223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.521688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.522140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.523066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:42.523668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:42.525234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6854b-2f63-4e9a-83d5-a245ff794be9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:42.539219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 475/953] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:42.540846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182f77-539b-419c-8279-711fa450040f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.647053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43182f77-539b-419c-8279-711fa450040f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:42.681776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:42.682429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed859c-85eb-4c80-92f1-b145135e617f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.682927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed859c-85eb-4c80-92f1-b145135e617f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.683477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed859c-85eb-4c80-92f1-b145135e617f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.688985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:42.688985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.688985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.688985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.689601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 476/954] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:42.691472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f7dcb-bacc-4fdc-9160-15fce40d250c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.692218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f7dcb-bacc-4fdc-9160-15fce40d250c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.692805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f7dcb-bacc-4fdc-9160-15fce40d250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.701443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:42.702917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:42.711103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-471f7dcb-bacc-4fdc-9160-15fce40d250c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:42.712604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-471f7dcb-bacc-4fdc-9160-15fce40d250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-502ca374-f247-4db8-b1d1-0a0c590e5e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:42.725502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:42.726446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:42.727418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:42.727418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:42.727418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:42.727418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:42.727418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 479/955] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:42.728348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be360e2-08b1-4be4-8359-b57842717b53 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.728633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be360e2-08b1-4be4-8359-b57842717b53 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.728908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be360e2-08b1-4be4-8359-b57842717b53 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.730296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 480/956] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:42.732952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be360e2-08b1-4be4-8359-b57842717b53 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.774551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:42.799316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 477/957] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:42.799316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.799316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.799316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.819112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:42.820339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:42.840243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be360e2-08b1-4be4-8359-b57842717b53 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:42.875647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:42.876598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:42.877055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:42.877500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:42.878412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:42.878918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:42.880401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:42.888920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 478/958] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:42.890358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.903172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 481/959] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:42.904812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:42.957002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.006163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aded9dd-bd83-4949-bdee-6f6551c1477e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.012887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b88b1-b7cd-4345-8dea-f1764ab25934 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.023902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.043750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.044896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.045176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.045460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.046480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.047947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.051806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.052761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.053046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.053318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:43.060433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:43.061514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:43.062392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.063203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.063203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.066627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 479/960] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:43.067536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:43.078881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:43.080079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 482/961] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:23:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:43.081063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.183921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e50c28-a03f-425a-b6a9-6acafb756eeb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.184117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832bedb-84c2-43da-8238-6226261b9e05 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.188792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.220359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.221169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.221448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.221722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.222366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.224244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.224365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.224924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.225190 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.225461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.226158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:43.226461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.227852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.232783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 483/962] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:43.233546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:43.237110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:43.238252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:43.239939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:43.242077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 480/963] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:43.242995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.292081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.348747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb74bbd-614d-4d64-beab-677bd8e76f72 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.350222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e75c-15cb-4832-a157-cf30ba3073c9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.386760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.387558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.387934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.388207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.388309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.388944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.389054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:43.389223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.389320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.389492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.390182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:43.390486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.390777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.391856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.397571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 481/964] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:43.398283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.421818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 484/965] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:43.428312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.471675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.549627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf186de-66f9-4d02-b0cc-7af022ef2cc7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.561355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547af2bd-8a2b-4673-b06e-01796f771759 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.563264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.595253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.595947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.596232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.596509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.597213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:43.597516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.598955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.604286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.605010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.605358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.605700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.608658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 482/966] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:43.614420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.683960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.685859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.714624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.741295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383ac7-744b-47d0-9159-79ba4be59dd8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.787419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.788367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.788655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.788933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.789645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:43.789953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.792257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.798672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 485/967] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:23:43.803420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 483/968] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:43.804460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.804879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.883693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.913280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89f66-8b11-49b1-a59a-70423741726c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.932485 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b83ce7-8706-43a7-8820-c44df6f510be None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.949417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.959510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.960661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.961143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.961638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.971784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.972433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.972650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.973554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.974306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.974916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.975703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.976811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:43.996708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:43.997921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:43.999106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:43.999106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:43.999106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:43.999106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:43.999372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 484/969] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:44.006634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 486/970] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:44.008373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.139665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:44.145935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626f7e0-389b-4e10-a58d-6ea24818ef8f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.183716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.184389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.184671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.184946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.185672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:44.185977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.187542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914ddc-db78-4eaf-835c-f51d17858c1c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.201094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 485/971] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:44.206746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.308766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea786c0-9818-4526-b290-6d2a7ba0ed51 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.309111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea786c0-9818-4526-b290-6d2a7ba0ed51 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.309318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea786c0-9818-4526-b290-6d2a7ba0ed51 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.310688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 486/972] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:44.315778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.316062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.316342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.325232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a733a0-43fd-4db0-9722-b6cbb2a002a6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.334621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:44.335946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:44.362062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.362859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.363143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.363504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.370585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.372058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:44.386258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:44.387245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:44.388617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 487/973] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:44.395779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.409948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 487/974] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:44.440797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.451035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:44.514401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb84f19-b9f9-4e19-9b1a-0847e408ea0e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.549205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:44.551382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.552075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.552332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.552607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.553616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:44.553737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.555192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.559935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb8a0a-f8d3-4fe9-83c3-6a45e238bde8 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.572914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 488/975] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:44.577283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.607435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.608442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.608893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.609274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.610220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.612379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.614032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING keystone.server.flask.common [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 09:23:44.630769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.sql.core [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:23:44.630769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 09:23:44.630769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [parameters: {'user_id': 'ba30bab3632e482996911aa9553c0317', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 09:23:44.630769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80184) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application result.read() Jul 20 09:23:44.653745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:23:44.654855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application yield resource Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise self.value Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application session.commit() Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 09:23:44.655868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self.session.flush() Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application compat.raise_( Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 09:23:44.656808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:23:44.658194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application result.read() Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:23:44.659259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ba30bab3632e482996911aa9553c0317', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:44.660932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 09:23:44.661992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 488/976] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Jul 20 09:23:44 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:44.668728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.697423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a3400-7750-4521-a0ff-d2b8b2bf0178 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.699157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:44.735830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.736493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.736777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.737052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.737882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] RBAC: Adding query filter params (role.id=c476d036502841398d096ac28f36b62e, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=1ab2924b8a9d41a7a6b74a73a7b809bd) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:44.738194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.740036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.746619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 489/977] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/role_assignments?user.id=1ab2924b8a9d41a7a6b74a73a7b809bd&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=c476d036502841398d096ac28f36b62e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:44.789961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8fed-767a-46e5-b9cd-766c26a26271 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:44.828407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:44.829144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:44.829424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:44.829695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:44.838902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:44.840606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-efceb0c7-e8cb-428e-b374-eeb9231a692f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:44.856356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:44.857434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:44.858850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 489/978] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:44.864101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:44.971323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94834d-f153-4a2a-9024-0dbb1a04946b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:45.007617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:45.008288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:45.008572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:45.008850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:45.009652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:45.009960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:45.011421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055849f-1aee-4139-9c5c-8d3af30e022e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:45.047747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 490/979] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Jul 20 09:23:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:45.641607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f844e-9294-463d-97fa-ee861e51e995 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:45.642121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f844e-9294-463d-97fa-ee861e51e995 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:45.642200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f844e-9294-463d-97fa-ee861e51e995 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:45.646506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 490/980] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:45.651537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f824a9-553e-48fc-9e23-f110951b2f7f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:45.651927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f824a9-553e-48fc-9e23-f110951b2f7f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:45.652111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f824a9-553e-48fc-9e23-f110951b2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:45.669158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-95f824a9-553e-48fc-9e23-f110951b2f7f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:45.670653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-95f824a9-553e-48fc-9e23-f110951b2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:45.686030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:45.741518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 491/981] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:45.763727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f844e-9294-463d-97fa-ee861e51e995 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:45.766973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:45.873427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f844e-9294-463d-97fa-ee861e51e995 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:45.907269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:45.907980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:45.908262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:45.908537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:45.909299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:45.911338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b3b2c-970f-4f39-93e7-cf971893f659 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:45.928360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 491/982] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:45 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:46.087945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b49d2-0c20-4483-924b-8fb8bb727bde None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.088253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b49d2-0c20-4483-924b-8fb8bb727bde None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.088503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b49d2-0c20-4483-924b-8fb8bb727bde None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.089881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 492/983] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:46.095403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.095699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.095975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.114817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:46.116411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:46.175234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:46.198650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 492/984] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:46.221220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b49d2-0c20-4483-924b-8fb8bb727bde None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:46.330734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b49d2-0c20-4483-924b-8fb8bb727bde None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:46.366195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:46.367374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.367879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.368359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.369197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:46.370753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:46.389588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:46.390650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:46.391568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:46.393033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 493/985] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:46.399472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:46.512984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5628-dc07-4eaa-84c8-e536e320bab9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:46.548640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:46.549335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.549621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.549897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.550600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:46.550909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:46.553070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:46.558194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 493/986] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:46.563720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:46.581849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:46.668725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313ad36-7a4a-45db-8262-710cf37d6b92 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:46.702500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:46.703167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.703491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.703767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.704474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:46.704781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:46.706154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:46.715868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 494/987] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:46.717384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:46.771526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:46.821173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ef888-f7af-47a9-a296-93f7a23cedfa None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:46.855185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:46.855868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.856154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.856430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.857229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] RBAC: Adding query filter params (interface=admin, service_id=23ddb4a819c44f959a630a4d934c0c86, region_id=RegionOne) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:46.857550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:46.858903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca933c-3927-4aaa-afa2-439ffa8d4024 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:46.865021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 494/988] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Jul 20 09:23:46 2022] GET /identity/v3/endpoints?service_id=23ddb4a819c44f959a630a4d934c0c86&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:46.870355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:46.888902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334df6-ac59-4af5-920f-b7ba34816c2d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.889190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334df6-ac59-4af5-920f-b7ba34816c2d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.889473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334df6-ac59-4af5-920f-b7ba34816c2d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.891723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 495/989] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:46.896695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:46.896987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:46.897266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:46.914926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:46.916153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:46.956836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:46.980425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e74336-8477-4b69-9aa9-122e527ce1a7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:46.986279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 496/990] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:47.008928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:47.019126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:47.019887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:47.020348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:47.020779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:47.021467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:47.022835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:47.029764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 495/991] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:46 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:23:47.071043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:47.131024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88164bb-91e0-4d43-8a6e-e733de6c3b05 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:47.167230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:47.168081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:47.168365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:47.168643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:47.169305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:47.170698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:47.184033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:47.185274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:47.186315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 497/992] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jul 20 09:23:47 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:47.193234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:47.302977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d47e6-fa24-4deb-89ab-4807c35db5e9 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:47.335885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:47.336549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:47.336833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:47.337108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:47.337807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:47.338114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:47.339513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395ac7c-439d-46d4-9a43-4049dfaf50e5 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:47.344420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 496/993] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:47.350634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:47.457984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271b0cc-b9f2-4db6-a8e4-6065a26acbb1 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:47.468674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:47.495584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:47.496252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:47.496530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:47.496802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:47.497500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] RBAC: Adding query filter params (type=image) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:47.497801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:47.499162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550d0b-9dc6-4175-b6e8-b51a40f96e71 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:47.511332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 498/994] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:48.005024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d831eb7-11a9-4228-bde9-083339f62cf0 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.005555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d831eb7-11a9-4228-bde9-083339f62cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.005555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d831eb7-11a9-4228-bde9-083339f62cf0 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.006993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 497/995] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:48.012096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.012483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.012644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.029665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:48.030844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:48.099426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 499/996] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:48.122050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d831eb7-11a9-4228-bde9-083339f62cf0 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:48.131533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:48.230709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d831eb7-11a9-4228-bde9-083339f62cf0 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:48.264076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:48.264793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.265072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.265343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.265990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:48.267379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:48.279503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:48.280514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:48.281428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:48.283446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 498/997] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:48.286804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:48.383400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb2bb-c1a5-46bd-bb72-1cc59946308a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:48.385665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:48.419842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:48.420496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.420780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.421052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.422009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:48.422312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:48.423707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:48.428898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 500/998] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:48.436777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:48.507703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:48.540299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb70c8-630c-49a3-90bb-2b3f539ba9ad None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:48.574283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:48.574946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.575227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.575527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.576227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:48.576533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:48.577897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:48.588102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 499/999] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:48.589577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:48.698016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a1ded-681e-4f21-8d0c-727777a05d29 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:48.734150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:48.734812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.735095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.735397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.736183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] RBAC: Adding query filter params (interface=public, service_id=d7f233d9dd354212899a51281bb8c6b8, region_id=RegionOne) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:48.736495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:48.737855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83ca1a-df3c-491c-b1e5-f4849698f480 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:48.748152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 501/1000] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:48 2022] GET /identity/v3/endpoints?service_id=d7f233d9dd354212899a51281bb8c6b8&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:48.749401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:48.851002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58c20b-d4c6-4345-80f3-83111ff5b535 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:48.885127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:48.885789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:48.886070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:48.886340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:48.887005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:48.888397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e40ada-94f7-4157-96fe-90ccf3f91f37 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:48.889419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:48.903317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 500/1001] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:48 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:49.101858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ba9-1046-455a-b639-37c098f8aa4b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.102353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ba9-1046-455a-b639-37c098f8aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.102353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4845ba9-1046-455a-b639-37c098f8aa4b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.103851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 502/1002] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:49.109079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.109468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.109647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.127532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:49.128702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:49.198616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 501/1003] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:49.220910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845ba9-1046-455a-b639-37c098f8aa4b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:49.252726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:49.325637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4845ba9-1046-455a-b639-37c098f8aa4b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:49.359058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:49.359804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.360079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.360345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.360987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:49.362353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:49.374846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:49.375867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:23:49.376737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:49.379375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 503/1004] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Jul 20 09:23:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:49.382152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:49.489578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e571eb7-c1a4-453e-92b9-03d1d0e1f19a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:49.516017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:49.524692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:49.525358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.525639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.525911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.526611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:49.526916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:49.528311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade630e-6bc5-4c6d-8b64-922d7ef7dd8b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:49.537940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 502/1005] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:49.539169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:49.560620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d822-3403-4ada-bc56-02211b4edf7a None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.560933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d822-3403-4ada-bc56-02211b4edf7a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.561179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0d822-3403-4ada-bc56-02211b4edf7a None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.562696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 503/1006] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:49.568232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.568518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.568795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.585463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:49.586645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:49.647619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:49.654799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f6923-98c6-4159-9eb8-1af76a54d59c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:49.656725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 504/1007] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:49.679847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:49.689000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:49.689673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.689957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.690234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.690939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:49.691252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:49.693254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:49.698304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 504/1008] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Jul 20 09:23:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:49.703700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:49.787241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67393bac-8587-4d8e-8185-ce86b72964c9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:49.810676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:49.812612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58055-7140-47c9-a1c4-e5c82d3a385b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:49.823248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:49.823936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.824223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.824498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.825230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:49.826597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddaf161-3cdd-41f6-bd08-9f1d7a14ed3f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:49.841691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 505/1009] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:49 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:49.861473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:49.862520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:49.863009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:49.863520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:49.864486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:49.866082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2001-7e58-48d1-9194-3edfd0987391 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:49.891365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 505/1010] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:23:50.974597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea351c-de2f-44c8-98ea-2d5ec8d5d9b8 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:50.975174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea351c-de2f-44c8-98ea-2d5ec8d5d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:50.975174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea351c-de2f-44c8-98ea-2d5ec8d5d9b8 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:50.976696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 506/1011] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:50.981855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e525-ef09-4990-901a-619057f76129 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:50.982304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e525-ef09-4990-901a-619057f76129 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:50.982515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888e525-ef09-4990-901a-619057f76129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.000335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c888e525-ef09-4990-901a-619057f76129 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:51.001539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c888e525-ef09-4990-901a-619057f76129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:51.013865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:51.083738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 506/1012] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:50 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:51.105662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea351c-de2f-44c8-98ea-2d5ec8d5d9b8 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:51.209986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea351c-de2f-44c8-98ea-2d5ec8d5d9b8 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:51.243904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:51.244637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.244921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.245196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.245843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:51.247207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:51.259050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:51.260161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:51.261174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 507/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:51.268006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e525-ef09-4990-901a-619057f76129 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:51.366162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888e525-ef09-4990-901a-619057f76129 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:51.399432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:51.400095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.400378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.400654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.401359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:51.401665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:51.403029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:51.411367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 507/1014] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:51.414608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:51.469222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:51.526738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdd1f9-a971-47e1-9ecd-21c14d4d4953 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:51.560721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:51.561380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.561660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.561931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.562636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:51.562941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:51.564329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62b708-81e0-4449-baab-9dab6fce91b6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:51.569935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 508/1015] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Jul 20 09:23:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:51.575467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:51.584956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:51.675742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcd251-401a-47f2-b81d-b0d097f2bd23 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:51.711591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:51.712250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.712530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.712801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.713546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:51.714927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed0737-3bc4-416b-8e3e-d672ac394d37 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:51.739316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 508/1016] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:51.938656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5940-8742-4d64-a1ac-bf766a2ccf1f None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.938902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5940-8742-4d64-a1ac-bf766a2ccf1f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.939140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5940-8742-4d64-a1ac-bf766a2ccf1f None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.940703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 509/1017] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:51.946427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.946718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.946999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.964292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:51.965747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:51.977932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cc45-e888-4307-90af-a1d4a6a44668 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.978540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cc45-e888-4307-90af-a1d4a6a44668 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.978819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cc45-e888-4307-90af-a1d4a6a44668 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:51.980217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 510/1018] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:51.985441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:51.985725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:51.986000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.008884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:52.010094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:52.032666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:52.052812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 509/1019] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:52.075928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.087344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 511/1020] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:52.107252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.150228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:52.187554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abbdc8-933b-464f-80f6-96c2f63d0fd6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.212007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec2b1a-f0d8-4134-af77-9cde24bc1e67 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.226030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.226812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.227092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.227401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] PATH_INFO: `/v3/services/image` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.228047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.229403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:52.241890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:52.242912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:52.244394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 510/1021] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:52.248136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.248928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.249028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.249207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.249479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.250625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.252175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.259612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 512/1022] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:52.260492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.304707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:52.352421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89d0c7-c175-4418-8115-9bc73bd2449d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.371314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0bae-edfc-4872-9674-5685288cad03 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.381477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:52.386014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.386677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.386956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.387227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.387952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] RBAC: Adding query filter params (name=image) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:52.388258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.389599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.394461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 511/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:52.401211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.416086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.416757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.417046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.417317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.418019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.419476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.459383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 513/1024] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:23:52 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:23:52.515092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9996189-9ffa-4e2c-b950-46c2299b253e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.533633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:52.556175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.556824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.557107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.557382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.558102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] RBAC: Adding query filter params (type=image) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:52.558411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.559842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.565588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 512/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:52.572151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.682713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d78fde-2c4e-4450-9103-95f7c49df362 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.716969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.717619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.717906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.718184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.718971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] RBAC: Adding query filter params (interface=public, service_id=17b0700622c54e46b250f4601025bb45, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:52.719338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.720711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9e2f9-1a86-4e1a-8a9e-3a1fc4a733e8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.726438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 514/1026] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/endpoints?service_id=17b0700622c54e46b250f4601025bb45&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:52.732180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:52.835787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf973ea-481d-429b-ab0f-d38966dcd151 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:52.869593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:52.870263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:52.870549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:52.870828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:52.871532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:52.872901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8164de-80ee-47d7-9dad-fcb2ed5f793f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:52.883325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 513/1027] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:23:52 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:53.892893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ee7e-136d-4edc-9243-8568e03a046b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:53.893448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ee7e-136d-4edc-9243-8568e03a046b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:53.893448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77ee7e-136d-4edc-9243-8568e03a046b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:53.895331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 515/1028] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:53.900776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:53.901182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:53.901350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:53.917961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:53.919135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:53.960801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:53.989907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 514/1029] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:54.013919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77ee7e-136d-4edc-9243-8568e03a046b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.040385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:54.120258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77ee7e-136d-4edc-9243-8568e03a046b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.154209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a27db-3488-4a50-8405-58651089a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.154939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a27db-3488-4a50-8405-58651089a100 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.155218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a27db-3488-4a50-8405-58651089a100 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.155518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a27db-3488-4a50-8405-58651089a100 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.166316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a27db-3488-4a50-8405-58651089a100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-296a27db-3488-4a50-8405-58651089a100 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:54.174774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:54.175918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:54.181867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 516/1030] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:54.182591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.283049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e08b0b-f61d-4667-9dbb-35385d6aea27 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.317607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.318572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.319046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.319542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.320459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:54.320944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.322492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.331001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 515/1031] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:54.336437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a27db-3488-4a50-8405-58651089a100 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.425664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:54.451650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a27db-3488-4a50-8405-58651089a100 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.492574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.493371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.493647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.493914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.506015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.507479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-21cc8e8a-1a41-43b7-824f-bbd5f47f1b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:54.525660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:54.526637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:54.527971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:54.527971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:54.527971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:54.527971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:54.531660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 517/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:54.532877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.547175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001489d-a402-4842-9259-1327f4b1f656 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.547845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001489d-a402-4842-9259-1327f4b1f656 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.548301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001489d-a402-4842-9259-1327f4b1f656 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.549842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 518/1033] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:54.555143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.555474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.555747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.571839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:54.573905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:54.575178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:54.644627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f7ff-4784-4155-8471-2e7da7accab0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.647489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 519/1034] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:54.671467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.685565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.686160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.686438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.686708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.687451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:54.687758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.689109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.706359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 516/1035] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:54.711762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.748228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:54.788716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cb017-aa8d-4a4f-be83-0d724248c5c2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.809988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:54.818132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37ad7e-70ed-4a54-8f3c-2524a8866ee6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.827102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.827916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.828194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.828463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.829649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.831066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.838381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 520/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:54.839209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.855572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.856234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.856510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.856812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.864376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.865934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:54.880816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:54.881793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:54.882769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 517/1037] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:54.888033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:54.948311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4a072-211c-4380-a147-70822dc5782f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:54.981746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:54.982416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:54.982700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:54.982972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:54.983716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:54.985148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca1e0b-daa9-40a9-a02c-972014f48ab0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:54.992731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.002652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a0618-c07c-4517-9122-c01d92ecbd67 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.006283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 521/1038] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:23:54 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:55.007608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288e8bf-4700-49fd-b937-00919e2c68bc None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.007900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288e8bf-4700-49fd-b937-00919e2c68bc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.008174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288e8bf-4700-49fd-b937-00919e2c68bc None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.009606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 522/1039] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:55.021791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.022126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.022362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.040233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.040920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.041202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.041475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.042366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:55.042677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.043336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:55.044177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.044572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:55.069276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 518/1040] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:55.070609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.088243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.133375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 523/1041] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:55.158185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.187389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dcfa3-de6e-47c9-8225-b497df017e58 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.227849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.228431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.228823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.228984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.275618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201599d-986c-46a4-a598-c44e25f4fb5d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.298631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.299227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.300109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.316248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.320541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.321620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.322120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.322608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] PATH_INFO: `/v3/services/image` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.323505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.325133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:55.338354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:55.339435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:23:55.340337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:55.341129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 524/1042] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:55.346844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.416580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 519/1043] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:23:55.418265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.460333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9b9a-ce9d-47da-b012-474779d7df5a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.495830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.496418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.496702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.496973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.497686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] RBAC: Adding query filter params (name=image) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:55.497994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.499408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.505698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 525/1044] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:55.511779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.535533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd7c84-9fe0-40b2-bfb3-9fc878ea6638 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.550061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.562906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.577109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.577908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.578190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.578465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.589644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.591230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:55.610402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:55.611402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:55.612333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:55.612333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:55.612333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:55.612333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:55.612333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 520/1045] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:55.617732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7b550-af70-429a-88df-c7582c928b26 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.619947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.655422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.656097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.656380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.656653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.657357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] RBAC: Adding query filter params (type=image) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:55.657681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.659029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.669121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 526/1046] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:55.670121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.728009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748c67-d1dd-4b0b-842b-fcac50539041 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.765842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.766620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.766898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.767268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.768014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:55.768319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.769672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.771233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ff1d8-4de0-4aa5-8914-dbce1ddd4b18 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.787669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.789696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 521/1047] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:55.790525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:55.808960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.809618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.809899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.810174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.810913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.812308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.819769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:55.833336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 527/1048] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:23:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:55.901078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbb109-9f19-4385-a81a-112300df48b2 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:55.945254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:55.946064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:55.946345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:55.946616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:55.954120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:55.955590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:23:55.971493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:55.972484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:23:55.973800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:55.975624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 522/1049] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:55.978836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:56.082976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9efce2-c2c5-4757-bb49-be1f649e2032 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:56.116337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:56.117278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:56.117758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:56.118207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:56.119156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:56.119662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:56.121245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95030e5-f252-462f-8f6c-0c1c829b7192 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:56.134380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 528/1050] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:56.140617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:56.226960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:56.246998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0bfd8-28f9-44b9-9018-d158a1003bda None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:56.281163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:56.281820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:56.282102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:56.282374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:56.283199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] RBAC: Adding query filter params (role.id=c476d036502841398d096ac28f36b62e, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:56.283535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:56.284958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c17ed-f62e-4eb9-93c3-698f2e57817e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:56.291005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 523/1051] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:56 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=c476d036502841398d096ac28f36b62e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:57.119159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4101d-9579-464c-a0f3-a994b801ec88 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.119705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4101d-9579-464c-a0f3-a994b801ec88 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.119785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4101d-9579-464c-a0f3-a994b801ec88 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.121146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 529/1052] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:57.126187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.126474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.126745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.144136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:57.145444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:57.211302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 524/1053] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:23:57.234784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4101d-9579-464c-a0f3-a994b801ec88 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:57.267758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:57.346705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4101d-9579-464c-a0f3-a994b801ec88 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:57.380727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:57.381440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.381720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.381994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.392306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:57.400009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:57.400944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:57.401836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 530/1054] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:23:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:57.409530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:57.496878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:57.521916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8066b-c53f-4e89-a661-2fb31e495fc2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:57.555564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:57.556514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.556978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.557433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.558354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:57.558830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:57.560375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:57.573825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 525/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:57.575093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:57.662691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:57.675613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458c08-a504-4ce6-a77a-ab58395e4292 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:57.710021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:57.711154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.711644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.712118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] PATH_INFO: `/v3/users/1bedcc787e094414b63a438adaacbd07` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.725418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] RBAC: Authorizing `identity:get_user(user_id=1bedcc787e094414b63a438adaacbd07)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:57.727036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc87c1-a6bf-41b7-b40b-ec8c0450cdd4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:57.728891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 531/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:23:57 2022] GET /identity/v3/users/1bedcc787e094414b63a438adaacbd07 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:57.734581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:57.834091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc596-2998-4946-8a77-26ee01459024 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:57.867131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:57.867803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.868079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.868348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.877016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:57.914260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832616-359b-424f-ba4e-03de0c4c3c89 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.914505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832616-359b-424f-ba4e-03de0c4c3c89 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.914830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5832616-359b-424f-ba4e-03de0c4c3c89 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.916401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 532/1057] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:23:57.921270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:57.922237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:57.922237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:57.939069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:57.940265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:57.940863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:57.942621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.015337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 533/1058] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:58.035150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.062343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 526/1059] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:23:58.068158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.106537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.121985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.146447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e510ec-0ebf-4c77-847b-28c6f64aeba2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.177767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b3bf9-2a43-475d-b9dc-bf36d96e4eaa None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.183767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.184423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.185115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.185407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.211631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.212437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.212719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.212989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.214311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.215615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.217635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 527/1060] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:58.223789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.271463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.273159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.333053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ffab0-c19d-4095-8daa-d0ebeada6e5a None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.340901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.356313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.369724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.370383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.370666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.370945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.371814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=8a2bc232f80f430296432fd54f9f7a96, user.id=1bedcc787e094414b63a438adaacbd07) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:58.372124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.373562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9247f7-9329-4736-aff8-9f116fb49dd4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.391314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 528/1061] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/role_assignments?user.id=1bedcc787e094414b63a438adaacbd07&scope.project.id=8a2bc232f80f430296432fd54f9f7a96&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:58.391314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cddbb6-229f-4b9f-befb-9d41d5307b71 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.391314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cddbb6-229f-4b9f-befb-9d41d5307b71 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.391314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cddbb6-229f-4b9f-befb-9d41d5307b71 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.391782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 534/1062] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:23:58.391873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 529/1063] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:23:58.391945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.392029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.392029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.392029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.409002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:58.410220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:58.497788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358792f-673e-4291-9005-eb209221fcbd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.501104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 530/1064] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:23:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:23:58.524283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.538390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.539198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.539504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.539780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.551987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.553680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:58.572484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:58.573430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:58.574337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:58.574337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:58.574337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:58.574337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:58.579759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 535/1065] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:58.581137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.611084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.637023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.640467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27469766-7fa4-40ce-b307-78f38b78a30f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.678730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.679586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.679868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.680140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.690542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:23:58.690779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a14a7c-249b-4418-bb74-eedc4e79804e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:23:58.699667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:58.700644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:23:58.701529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 531/1066] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:58.706667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.734390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.735332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.735618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.735892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.736614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:58.736926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.738286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.750675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 536/1067] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:58.825941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ad25-f499-4d86-8d6b-8d9c02f485ba None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:58.863356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:58.864055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:58.864335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:58.864613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:58.865368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:58.865673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:58.867077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:58.876515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 532/1068] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:23:58.882819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:58.976824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:58.990542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe46ec1-efeb-4cfa-a82e-01d31a371b07 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.028344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.029233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.029614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.029895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.043217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.043217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:59.060823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:59.061869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:59.062806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:59.062806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:59.062806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:59.062806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:59.062806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 537/1069] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:23:59.069009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.093897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:59.168427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef2e2c-4cea-4086-ae90-01bfbe9f8cc7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.201511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.202167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.202447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.202719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.203459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:59.203769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.205119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.217386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 533/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:59.223337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.322861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af302-7b42-49ff-ba11-d6e995b678bb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.355835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.356621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.356901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.357172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.364690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.366109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.368574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:23:59.380616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:59.381852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:59.382855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 538/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:59.387571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.468845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:59.487677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec8ba1-07a0-49f3-9bc3-0cad3d492951 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.523637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.524315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.524600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.524876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.525716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:59.526027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.527484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.544423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 534/1072] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:59.550341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.648534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0329-da0b-480c-9a83-bd918b22f78c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.681861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.682645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.682977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.683254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.693535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.694968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:23:59.711681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:23:59.712735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:23:59.713968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:23:59.713968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:23:59.713968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:23:59.713968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:23:59.713968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 539/1073] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:23:59.718567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.814738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0498-7a14-4dd2-99ef-f2aaad3a4a03 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:59.848961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:59.849611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:59.849891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:59.850165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:59.850875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:59.851179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:59.852564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:59.860616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:59.864997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 535/1074] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:59.870417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:59.923954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:59.973928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e942c2d-b2b0-4c0b-b6c5-36ab67a775b4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.007989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:00.008772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.009052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.009323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.015794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:00.017215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:24:00.031610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:00.032512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:00.033467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 540/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:00.038762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:00.135818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ce999-0a8b-43cd-ba85-24d5deeabe1c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.167642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:00.168293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.168573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.168840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.169610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:00.169915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:00.171341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e9ebc3-db4d-4b0f-bb46-db1e851000ae None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:00.183883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 536/1076] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:24:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:00.189690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:00.288454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a06812-b25b-408f-a9de-d8c99b4ca8e7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.302476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:00.323215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:00.324131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.324413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.324687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/c476d036502841398d096ac28f36b62e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.326875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=0364cf432f5b44a9a62f08c4243fff5e, role_id=c476d036502841398d096ac28f36b62e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:00.328344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c30904-6431-4638-9b4a-63149cc8c247 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:00.341171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 541/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:24:00 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/0364cf432f5b44a9a62f08c4243fff5e/roles/c476d036502841398d096ac28f36b62e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:24:00.425099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15eb6e-5248-4227-82ff-ccde55c874e5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.425652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15eb6e-5248-4227-82ff-ccde55c874e5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.425652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15eb6e-5248-4227-82ff-ccde55c874e5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.427912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 537/1078] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:00.433583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.433949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.434135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.452240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:00.453467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:00.526335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 542/1079] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:00.549295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15eb6e-5248-4227-82ff-ccde55c874e5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:00.617279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:00.659017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15eb6e-5248-4227-82ff-ccde55c874e5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.694979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:00.695782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.696059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.696327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.706241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:00.714508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:00.715517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:00.716411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 538/1080] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:00.721758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:00.821101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151f6f-9c8f-4386-897b-ff0e5b3854f5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.822603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:00.856234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:00.856899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:00.857181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:00.857456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:00.858198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:00.858504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:00.859899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:00.868612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 543/1081] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:00.874344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:00.972798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c506da-6dd7-4bd8-9408-36bf9e57b386 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:00.992583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:01.006718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:01.007641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:01.007921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:01.008193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] PATH_INFO: `/v3/users/1bedcc787e094414b63a438adaacbd07` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:01.009371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=1bedcc787e094414b63a438adaacbd07)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:01.010789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:01.012456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 539/1082] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:00 2022] GET /identity/v3/users/1bedcc787e094414b63a438adaacbd07 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:01.018280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:01.115381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198fd7d-97f6-4a3b-897f-c02ee64983d9 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:01.150168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:01.151646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:01.152118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:01.152576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:01.154464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:01.156996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8890eb0-c0ff-4735-9582-905ddb19c610 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:01.160108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 544/1083] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:01 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:01.166706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:01.264915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454aa8c-5fd4-4387-b5d7-25101d897bd6 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:01.299213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:01.300128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:01.300410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:01.300684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96/users/1bedcc787e094414b63a438adaacbd07/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:01.303343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a2bc232f80f430296432fd54f9f7a96, user_id=1bedcc787e094414b63a438adaacbd07, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:01.304780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721038cd-2f91-4505-bb38-465db3b39cc4 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:01.319212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 540/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:24:01 2022] PUT /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96/users/1bedcc787e094414b63a438adaacbd07/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:02.484580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3456-390e-4daa-adce-bcca652f0ded None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:02.485108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3456-390e-4daa-adce-bcca652f0ded None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:02.485108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb3456-390e-4daa-adce-bcca652f0ded None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:02.486532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 545/1085] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:02.492059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:02.492516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:02.492614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:02.509834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:02.511023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:02.556952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:02.581791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 541/1086] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:02.606747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb3456-390e-4daa-adce-bcca652f0ded None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:02.637578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:02.705838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb3456-390e-4daa-adce-bcca652f0ded None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:02.739513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:02.740311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:02.740594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:02.740864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:02.750486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:02.759212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:02.760408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:02.761537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 546/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Jul 20 09:24:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:02.766433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:02.865109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af53a8-86f1-42b7-a9ea-f18cefbb9271 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:02.898186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:02.898845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:02.899123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:02.899446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:02.900181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:02.900483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:02.901840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:02.909625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 542/1088] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:24:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:02.915946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.003717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.012408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884e03c-7fa9-4924-b22c-9938caae822e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.045562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.046379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.046657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.046924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.057874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.059338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:24:03.075989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:03.076919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:03.077852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:24:03.077852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:24:03.077852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:24:03.077852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:03.077852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 547/1089] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:03.083643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.124954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.184621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fc80c-238a-4a44-a789-b81d861a4440 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.218028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.218685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.218965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.219235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.219975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:03.220279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.221620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.233450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 543/1090] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:03.239071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.335449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463b6f2-f684-4b4f-8ace-054d9c4f9170 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.369947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.370740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.371023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.371311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.378764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.380846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-6a257274-8432-4d5b-998f-b195f09126aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:24:03.394566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:03.396220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:03.397195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 548/1091] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:03.402868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.461353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f9a7-0af9-4f9b-b004-645336b29257 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.461640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f9a7-0af9-4f9b-b004-645336b29257 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.461912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19f9a7-0af9-4f9b-b004-645336b29257 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.463303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 549/1092] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:24:03.468234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.468523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.468802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.474828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.486048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:03.487248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:03.503835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.507212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d69e3-617d-463c-aa3e-c9348e1c4629 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.541458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.542121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.542401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.542676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.543530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:03.543842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.545262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.555963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 550/1093] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:24:03.566748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 544/1094] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:03.568878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.579219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.667314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872b0f2-597a-406c-b708-734c665a8960 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.675452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ef28c-3385-49ab-a28e-7214d5a9e3b0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.701860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.702506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.702791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.703067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.705704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.710301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.711092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.711408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.711688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.722304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:03.730116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:03.731089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:03.732025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 545/1095] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:24:03.737394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.757912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.771521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.772977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.837818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979622b4-190e-4112-8a1d-733535c9f178 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.870277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 551/1096] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:03.870841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:03.871531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:03.871808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:03.872081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:03.872814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:03.873117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:03.874467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:03.876112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.884583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 546/1097] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:03.889991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:03.909778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.967750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:03.975705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ff16-1a1a-413b-83f7-d1ebc5db39f3 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:03.988660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae11dad-3ec7-4dfc-b918-119b8de4e3ab None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.008846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.009632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.009913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.010187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.019957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.022066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.022910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.023815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.024163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.024370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] PATH_INFO: `/v3/users/1bedcc787e094414b63a438adaacbd07` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.025573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] RBAC: Authorizing `identity:get_user(user_id=1bedcc787e094414b63a438adaacbd07)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.026980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.028654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 547/1098] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/users/1bedcc787e094414b63a438adaacbd07 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:04.034244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:24:04.040984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:04.042045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:04.042994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:24:04.042994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:24:04.042994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:24:04.042994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:04.042994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 552/1099] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:24:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:04.048009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.134568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f836d-ead1-4732-bf12-9f021342900e None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.137446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.154829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3826f3-7f25-426f-85de-5059dae30f36 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.168067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.168714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.168994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.169266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.186480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.188641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.189365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.189709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.190041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.190845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:04.191152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.193152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.204695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 553/1100] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:04.210853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.236247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.237608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.307500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ece435-8e24-45cb-8996-08fd9850b0e6 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.333138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 548/1101] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:24:04.338918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.340433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.341224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.341506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.341779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.348495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.350100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:24:04.364934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:04.366513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:04.368336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 554/1102] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:24:04.373342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.382350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.391610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.443840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7454e2-a1cd-4937-bf23-64337f930fa9 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.477504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66decb85-32c9-479b-a8ad-c31415c932a7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.478650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.479543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.479857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.480133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.481321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.482753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.484572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 549/1103] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:04.490562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.512045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.512696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.512974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.513247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.514020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:04.514328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.515785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.527571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 555/1104] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:04.533565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.591797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602818-78ee-490d-8937-efe5e0c1580d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.597512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.610902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.630473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.631128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.631444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.631718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.632532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=8a2bc232f80f430296432fd54f9f7a96, user.id=1bedcc787e094414b63a438adaacbd07) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:04.632839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.634254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b5261-d859-4dcd-9457-5b1f125de0c1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.634346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1cd10-dd47-4110-a18e-f3061f8ab627 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.639976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 550/1105] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/role_assignments?user.id=1bedcc787e094414b63a438adaacbd07&scope.project.id=8a2bc232f80f430296432fd54f9f7a96&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:04.668843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.669523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.669810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.670088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.670931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] RBAC: Adding query filter params (role.id=c476d036502841398d096ac28f36b62e, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=0364cf432f5b44a9a62f08c4243fff5e) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:04.671246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.673011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfe93f-9c34-4110-835b-a0cc56d0cdb7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.691331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 556/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:04 2022] GET /identity/v3/role_assignments?user.id=0364cf432f5b44a9a62f08c4243fff5e&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=c476d036502841398d096ac28f36b62e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:06.695816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c9119-0d51-4716-aa7f-9f2af7c436c1 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:06.696372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c9119-0d51-4716-aa7f-9f2af7c436c1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:06.696372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c9119-0d51-4716-aa7f-9f2af7c436c1 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:06.697690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 551/1107] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:06.702859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:06.703329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:06.703437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:06.719747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:06.720905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:06.784114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 557/1108] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:06.808966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c9119-0d51-4716-aa7f-9f2af7c436c1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:06.908971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c9119-0d51-4716-aa7f-9f2af7c436c1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:06.941812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:06.943151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:06.943448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:06.943718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:06.944875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:06.946353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2395d-4879-43fa-baae-0adff7bf6530 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:06.947933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 552/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:06.953766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:07.010152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:07.049613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911872e5-d7d1-4136-aa84-c50a2358de17 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:07.082151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:07.083133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:07.083348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:07.083590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:07.084292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:07.085708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ab1e7-9467-4765-81fc-cadd5048de74 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:07.106329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 558/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:24:06 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:24:09.148708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcdba4-4a42-4943-a51b-498792fc4b0c None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.149255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcdba4-4a42-4943-a51b-498792fc4b0c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.149255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcdba4-4a42-4943-a51b-498792fc4b0c None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:09.150781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 553/1111] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:09.155830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.156264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.156420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:09.173183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:09.174368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:09.244157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 559/1112] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:09.267169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcdba4-4a42-4943-a51b-498792fc4b0c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:09.319940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:09.371730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcdba4-4a42-4943-a51b-498792fc4b0c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:09.404661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:09.405439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.405718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.405986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] PATH_INFO: `/v3/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:09.407453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] RBAC: Authorizing `identity:get_role(role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:09.408809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:09.410162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 554/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:09 2022] GET /identity/v3/roles/97a9c200717c42a4a09a99543c15fa03 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:09.416583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:09.461845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:09.513370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d6ec2-9da3-4d1d-b3d2-c41770e6b8f7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:09.545529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:09.546384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.546665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.546936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] PATH_INFO: `/v3/users/4a544808369545678e7526925330b9a0` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:09.560401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] RBAC: Authorizing `identity:get_user(user_id=4a544808369545678e7526925330b9a0)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:09.561830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:09.563502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 560/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:09 2022] GET /identity/v3/users/4a544808369545678e7526925330b9a0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:09.569412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:09.665685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa34f2c-d82c-4bf2-b9f9-8bf67348ccd8 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:09.681586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:09.699182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:09.699844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.700126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.700398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:09.765203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:09.766638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:09.860743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 555/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:09.866842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:09.963979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b0a54-e22f-4a03-bc32-5c0994cc5735 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:09.966148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:09.997757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:09.998599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:09.998881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:09.999158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:10.000356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:10.001780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f5525-6285-4b92-810a-fd2febfbf174 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:10.003490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 561/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:09 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:10.010196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:10.027841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:10.106345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c19a-5a7b-4b9c-b928-a14c3604c2d3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:10.138974 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:10.139644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:10.139926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:10.140199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:10.141015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=8a2bc232f80f430296432fd54f9f7a96, user.id=4a544808369545678e7526925330b9a0) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:10.141325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:10.142750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a28a-c3f4-4081-a78f-6208870720c1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:10.148404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 556/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:10 2022] GET /identity/v3/role_assignments?user.id=4a544808369545678e7526925330b9a0&scope.project.id=8a2bc232f80f430296432fd54f9f7a96&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:12.135853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067fc18-fad8-4890-9614-45fe613ae047 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.136406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067fc18-fad8-4890-9614-45fe613ae047 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.136406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067fc18-fad8-4890-9614-45fe613ae047 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.138130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 562/1118] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:12.143256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.143666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.143832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.160650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:12.162367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:12.227846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 557/1119] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:12.249913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067fc18-fad8-4890-9614-45fe613ae047 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:12.348971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067fc18-fad8-4890-9614-45fe613ae047 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:12.381229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:12.382059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.382347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.382619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] PATH_INFO: `/v3/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.384123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] RBAC: Authorizing `identity:get_role(role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:12.385485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:12.386906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 563/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:12 2022] GET /identity/v3/roles/97a9c200717c42a4a09a99543c15fa03 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:12.392746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:12.404519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:12.489432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc761c8-34f2-479e-8766-44e224790349 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:12.522453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:12.523334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.523618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.523891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] PATH_INFO: `/v3/users/4a544808369545678e7526925330b9a0` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.525068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] RBAC: Authorizing `identity:get_user(user_id=4a544808369545678e7526925330b9a0)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:12.526494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:12.528148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 558/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:12 2022] GET /identity/v3/users/4a544808369545678e7526925330b9a0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:12.534010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:12.574921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:12.631515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbd9e3-0105-4851-9694-2a9f65bd7c0d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:12.664069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:12.664852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.665126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.665394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.666544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:12.667991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c065c-8b62-4478-8553-b05fd1914db7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:12.669664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 564/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:12 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:12.676249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:12.752450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:12.773963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b7d04-ddc6-4050-902a-4f59b86ff768 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:12.807084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:12.807986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:12.808266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:12.808538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96/users/4a544808369545678e7526925330b9a0/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:12.810719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] RBAC: Authorizing `identity:create_grant(project_id=8a2bc232f80f430296432fd54f9f7a96, user_id=4a544808369545678e7526925330b9a0, role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:12.812167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf39024-5d55-46d3-83bb-998716e0c2fb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:12.825859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 559/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:24:12 2022] PUT /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96/users/4a544808369545678e7526925330b9a0/roles/97a9c200717c42a4a09a99543c15fa03 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:14.833192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65be5-eafd-4dd0-b1cd-c6909a1a9d57 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:14.833730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65be5-eafd-4dd0-b1cd-c6909a1a9d57 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:14.833730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f65be5-eafd-4dd0-b1cd-c6909a1a9d57 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:14.835095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 565/1124] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:14.840681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:14.841071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:14.841241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:14.857677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:14.858845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:14.922609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 560/1125] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:14.946797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f65be5-eafd-4dd0-b1cd-c6909a1a9d57 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:15.044982 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f65be5-eafd-4dd0-b1cd-c6909a1a9d57 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:15.054168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:15.078461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:15.079321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:15.079601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:15.079870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] PATH_INFO: `/v3/roles/97a9c200717c42a4a09a99543c15fa03` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:15.081309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] RBAC: Authorizing `identity:get_role(role_id=97a9c200717c42a4a09a99543c15fa03)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:15.083328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:15.084655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 566/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:14 2022] GET /identity/v3/roles/97a9c200717c42a4a09a99543c15fa03 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:15.090384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:15.153642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:15.187473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0778-ced9-4109-94cb-e7ba398cae31 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:15.222379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:15.223245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:15.223600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:15.223882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] PATH_INFO: `/v3/users/4a544808369545678e7526925330b9a0` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:15.225057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] RBAC: Authorizing `identity:get_user(user_id=4a544808369545678e7526925330b9a0)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:15.226480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:15.228155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 561/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:15 2022] GET /identity/v3/users/4a544808369545678e7526925330b9a0 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:15.234010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:15.329602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc2631-d80c-431d-8e6c-30d7cb5e6f8b None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:15.361968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:15.362528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:15.362806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:15.363078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:15.406095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:15.428086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:15.429509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:15.523991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 567/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:15.529960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:15.626134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5235ea5-e1f5-4d49-9c77-efcb8fb4f312 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:15.658536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:15.659413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:15.659702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:15.659978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] PATH_INFO: `/v3/projects/8a2bc232f80f430296432fd54f9f7a96` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:15.661161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] RBAC: Authorizing `identity:get_project(project_id=8a2bc232f80f430296432fd54f9f7a96)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:15.662575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8c192-1846-4af9-aa7d-02cc43edb7ce None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:15.664281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 562/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:15 2022] GET /identity/v3/projects/8a2bc232f80f430296432fd54f9f7a96 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:15.670303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:15.743790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:15.766537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b2c9d-c919-430e-a850-d6cfbfa39771 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:15.798867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:15.799527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:15.799807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:15.800075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:15.800884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] RBAC: Adding query filter params (role.id=97a9c200717c42a4a09a99543c15fa03, scope.project.id=8a2bc232f80f430296432fd54f9f7a96, user.id=4a544808369545678e7526925330b9a0) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:15.801189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:15.803116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb6cf4b-3e26-4814-b27f-7526a7740b6f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:15.808723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 568/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:15 2022] GET /identity/v3/role_assignments?user.id=4a544808369545678e7526925330b9a0&scope.project.id=8a2bc232f80f430296432fd54f9f7a96&role.id=97a9c200717c42a4a09a99543c15fa03 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:17.859722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b45db-7ec9-447f-aa50-7111d614b382 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:17.860316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b45db-7ec9-447f-aa50-7111d614b382 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:17.860316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b45db-7ec9-447f-aa50-7111d614b382 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:17.861626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 563/1131] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:17.867397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:17.867848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:17.867949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:17.884892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:17.886076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:17.950982 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 569/1132] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:17.973552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b45db-7ec9-447f-aa50-7111d614b382 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:17.977297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:18.071901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b45db-7ec9-447f-aa50-7111d614b382 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:18.104168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:18.104936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:18.105213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:18.105481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:18.106624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:18.108059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c292a3-1244-44f8-9226-5538c0517f79 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:18.109642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 564/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:24:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:18.116175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:18.182397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:18.212607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bd15e-8485-40d1-9a87-3e60108f0d44 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:18.245637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:18.246279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:18.246558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:18.246831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:18.247576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:18.249000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a8825-d05d-47ba-a204-375904e8b014 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:18.273671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 570/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:24:18 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:24:20.322375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40a8d-0112-4e50-8189-5742c1025ea7 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:20.322934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40a8d-0112-4e50-8189-5742c1025ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:20.322934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d40a8d-0112-4e50-8189-5742c1025ea7 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:20.324329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 565/1135] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:20.329572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:20.329993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:20.330133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:20.347268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:20.348560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:20.413025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 571/1136] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:20.437134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d40a8d-0112-4e50-8189-5742c1025ea7 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:20.506147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:20.535402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d40a8d-0112-4e50-8189-5742c1025ea7 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:20.567952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:20.568754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:20.569033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:20.569302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:20.570469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:20.571919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfa3b7-3e09-41ab-a653-4f4a20e4fc78 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:20.573561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 566/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:24:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:20.579276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:20.629589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:20.674784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d31f-833d-49d0-b687-6bd99a45e535 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:20.707000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:20.707677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:20.708000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:20.708348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:20.709054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:20.710473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3236-f78c-406f-9a8f-0966211696a0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:20.730307 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 572/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:24:20 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:24:22.784601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0846-4123-4844-9182-465cc5776471 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:22.785134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0846-4123-4844-9182-465cc5776471 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:22.785134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0846-4123-4844-9182-465cc5776471 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:22.787732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 567/1139] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:22.792856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:22.793280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:22.793640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:22.812005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:22.813193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:22.877932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 573/1140] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:22.900745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b0846-4123-4844-9182-465cc5776471 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:22.998484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b0846-4123-4844-9182-465cc5776471 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.029959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.030731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.031012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.031281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.039142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:23.042449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:23.050838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:23.052808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:23.054104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 568/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:23.058964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:23.097646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:23.156173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eac7c-c91b-4a29-ae6b-309f28277bb7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.189293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.189940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.190218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.190494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.191220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:23.191549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:23.193233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:23.201083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 574/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:23.206902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:23.302590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82ae9-98d1-482d-b719-2ba50ec2140b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.334686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.335570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.335852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.336126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] PATH_INFO: `/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.349282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] RBAC: Authorizing `identity:get_user(user_id=e722f92c0eb74bdfbbc1a1cdcfab25c3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:23.350696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:23.352348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 569/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:23 2022] GET /identity/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:23.357984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:23.452728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:23.455889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd7a-59af-4900-872c-ef25593673be None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.489891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.490531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.490810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.491084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.556587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:23.557961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:23.647143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:23.650862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 575/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:23.656669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:23.712720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:23.753340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0eb1-1eb2-4e52-bade-569f2086f767 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.785379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.786799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.787254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.787418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] PATH_INFO: `/v3/projects/b5f2733deeec454cb4c3ff80341263eb` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.788595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] RBAC: Authorizing `identity:get_project(project_id=b5f2733deeec454cb4c3ff80341263eb)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:23.790012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ec896-1f61-478e-b062-85c41bfe1dad None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:23.791716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 570/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:23 2022] GET /identity/v3/projects/b5f2733deeec454cb4c3ff80341263eb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:23.797707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:23.894027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aaf08-fa1f-4b60-9d75-d3a406700a48 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:23.925748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:23.926381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:23.927160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:23.927464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:23.928274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=b5f2733deeec454cb4c3ff80341263eb, user.id=e722f92c0eb74bdfbbc1a1cdcfab25c3) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:23.928582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:23.929986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d38e6-e3bf-440c-afc9-ee95e246ebc0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:23.935340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 576/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:23 2022] GET /identity/v3/role_assignments?user.id=e722f92c0eb74bdfbbc1a1cdcfab25c3&scope.project.id=b5f2733deeec454cb4c3ff80341263eb&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:25.944644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86fec-7827-4b7e-8b96-e5cb714050f3 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:25.945212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86fec-7827-4b7e-8b96-e5cb714050f3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:25.945212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab86fec-7827-4b7e-8b96-e5cb714050f3 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:25.946560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 571/1147] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:25.951797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926262a-78aa-45f2-854e-76167b960044 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:25.952190 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926262a-78aa-45f2-854e-76167b960044 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:25.952342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926262a-78aa-45f2-854e-76167b960044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:25.968538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2926262a-78aa-45f2-854e-76167b960044 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:25.969737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2926262a-78aa-45f2-854e-76167b960044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:26.026237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:26.034522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 577/1148] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:26.057975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab86fec-7827-4b7e-8b96-e5cb714050f3 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:26.155689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab86fec-7827-4b7e-8b96-e5cb714050f3 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:26.180756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:26.189284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:26.190063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:26.190342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:26.190611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:26.200552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:26.208685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:26.209659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:26.210523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 572/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:26.216073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926262a-78aa-45f2-854e-76167b960044 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:26.311501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926262a-78aa-45f2-854e-76167b960044 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:26.352147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:26.352803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:26.353082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:26.353353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:26.354089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:26.354391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:26.355791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:26.363588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 578/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:26.369551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:26.467174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d985e1-12a8-4271-9620-a64645d3fc52 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:26.498878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:26.499766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:26.500097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:26.500372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] PATH_INFO: `/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:26.501555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] RBAC: Authorizing `identity:get_user(user_id=e722f92c0eb74bdfbbc1a1cdcfab25c3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:26.502969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:26.504617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 573/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:26 2022] GET /identity/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:26.511123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:26.565868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:26.609658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df454e-29a9-4d14-9ac4-f78141f51617 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:26.651994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:26.652842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:26.653119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:26.653389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] PATH_INFO: `/v3/projects/b5f2733deeec454cb4c3ff80341263eb` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:26.654558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] RBAC: Authorizing `identity:get_project(project_id=b5f2733deeec454cb4c3ff80341263eb)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:26.656010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06b7bd-6f6f-46a1-8931-2ec491ad0540 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:26.657700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 579/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:26 2022] GET /identity/v3/projects/b5f2733deeec454cb4c3ff80341263eb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:26.663807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:26.717693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:26.761097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688ea5c-6185-4333-918e-794f70913552 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:26.793118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:26.793991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:26.794269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:26.794542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] PATH_INFO: `/v3/projects/b5f2733deeec454cb4c3ff80341263eb/users/e722f92c0eb74bdfbbc1a1cdcfab25c3/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:26.796491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5f2733deeec454cb4c3ff80341263eb, user_id=e722f92c0eb74bdfbbc1a1cdcfab25c3, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:26.797907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977ba00-5abc-4059-95ef-97d6eff4b6b1 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:26.810719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 574/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:24:26 2022] PUT /identity/v3/projects/b5f2733deeec454cb4c3ff80341263eb/users/e722f92c0eb74bdfbbc1a1cdcfab25c3/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:28.796119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aca5c-58d0-4dcd-8414-76a29b99aee9 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:28.796704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aca5c-58d0-4dcd-8414-76a29b99aee9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:28.796704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628aca5c-58d0-4dcd-8414-76a29b99aee9 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:28.798035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 580/1154] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:28.804471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:28.804906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:28.805013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:28.821887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:28.823067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:28.887383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 575/1155] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:28.911070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aca5c-58d0-4dcd-8414-76a29b99aee9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.009686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628aca5c-58d0-4dcd-8414-76a29b99aee9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.032480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:29.043502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.044281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.044564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.044834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.055045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:29.062601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:29.063745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:29.064951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 581/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:29.070441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.113563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:29.165721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49982d9d-5b41-4b28-932d-961a0a2452c3 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.198829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.199506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.199786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.200062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.200787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:29.201093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:29.202438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:29.210582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 576/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:29.216268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.312274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228948-3f73-4f1c-85f2-e88df46fe16a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.344455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.345313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.345598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.345990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] PATH_INFO: `/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.347174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] RBAC: Authorizing `identity:get_user(user_id=e722f92c0eb74bdfbbc1a1cdcfab25c3)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:29.348632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:29.350385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 582/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:29 2022] GET /identity/v3/users/e722f92c0eb74bdfbbc1a1cdcfab25c3 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:24:29.356140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.452659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29218e7e-222d-4012-88d9-825884bd16fd None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.459468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:29.485867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.486510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.486793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.487067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.551567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:29.553565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:29.646912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 577/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:29.652771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.704657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:29.751520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b7d17-a40a-4017-a366-39cc682cd040 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.783336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.784187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.784467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.784738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] PATH_INFO: `/v3/projects/b5f2733deeec454cb4c3ff80341263eb` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.785908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] RBAC: Authorizing `identity:get_project(project_id=b5f2733deeec454cb4c3ff80341263eb)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:29.788023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ae62b-c500-45e4-a180-d20fce06d806 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:29.789714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 583/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:29 2022] GET /identity/v3/projects/b5f2733deeec454cb4c3ff80341263eb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:29.795704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:29.803735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:29.889454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532a623-f01e-44d3-914e-5404c25a38f0 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:29.922001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:29.922575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:29.922851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:29.923121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:29.924215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=b5f2733deeec454cb4c3ff80341263eb, user.id=e722f92c0eb74bdfbbc1a1cdcfab25c3) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:29.924522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:29.925934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32de928-d0e0-4bee-adab-b92528626325 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:29.931779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 578/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:29 2022] GET /identity/v3/role_assignments?user.id=e722f92c0eb74bdfbbc1a1cdcfab25c3&scope.project.id=b5f2733deeec454cb4c3ff80341263eb&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:31.985877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aa929-d409-4b21-93e0-9365d196a9a6 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:31.986415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aa929-d409-4b21-93e0-9365d196a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:31.986415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aa929-d409-4b21-93e0-9365d196a9a6 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:31.987843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 584/1162] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:31.993561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:31.993964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:31.994113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:32.010324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:32.011559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:32.071673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:32.076440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 579/1163] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:32.099168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9aa929-d409-4b21-93e0-9365d196a9a6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:32.198103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9aa929-d409-4b21-93e0-9365d196a9a6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:32.218297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:32.232228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:32.232861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:32.233136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:32.233407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:32.296935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:32.298357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:32.392472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 585/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:32.401590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:32.497706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1d4be-0990-4990-a8a0-cb9aff4d6794 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:32.529426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:32.530208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:32.530486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:32.530754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:32.537647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:32.539060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-13ca2014-afc7-4a26-88ce-d5a96f1af374 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:32.553719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:32.554926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 09:24:32.556030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:32.559474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 580/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:24:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:32.560874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:32.581255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:32.656846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033959d-da02-4f7b-b6be-3a75f727b7ee None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:32.690839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:32.691517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:32.691802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:32.692077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:32.692831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:32.693138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:32.694502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e5ea4-aa57-4d08-a9fa-958fcce84138 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:32.701492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 586/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Jul 20 09:24:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:34.727586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68ebf4-76bc-4688-a06f-2adea86fc040 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:34.728091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68ebf4-76bc-4688-a06f-2adea86fc040 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:34.728091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68ebf4-76bc-4688-a06f-2adea86fc040 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:34.729445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 581/1167] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:34.734671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb25c-46b5-4e5d-acb8-73449e0a2e45 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:34.735042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb25c-46b5-4e5d-acb8-73449e0a2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:34.735230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb25c-46b5-4e5d-acb8-73449e0a2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:34.751763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f7fcb25c-46b5-4e5d-acb8-73449e0a2e45 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:34.752944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f7fcb25c-46b5-4e5d-acb8-73449e0a2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:34.819135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 587/1168] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:34.842354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68ebf4-76bc-4688-a06f-2adea86fc040 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:34.902063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:34.942312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68ebf4-76bc-4688-a06f-2adea86fc040 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:34.975367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:34.976021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:34.976306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:34.976583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:34.977326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:34.978732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d49b5-85f1-4ce6-8140-2b1df418c7e9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:34.995223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 582/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Jul 20 09:24:34 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:24:37.037906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56ef6-2adc-46a5-9d63-30d6e445d274 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:37.038550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56ef6-2adc-46a5-9d63-30d6e445d274 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:37.038550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba56ef6-2adc-46a5-9d63-30d6e445d274 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:37.039878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 588/1170] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:37.044940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:37.045369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:37.045476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:37.062289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:37.063490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:37.127350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 583/1171] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:37.151509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56ef6-2adc-46a5-9d63-30d6e445d274 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:37.166832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:37.250315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba56ef6-2adc-46a5-9d63-30d6e445d274 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:37.283466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:37.284321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:37.284600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:37.284870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] PATH_INFO: `/v3/domains/f54cfe2306104b0dae52c1c19486516e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:37.295090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] RBAC: Authorizing `identity:get_domain(domain_id=f54cfe2306104b0dae52c1c19486516e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:37.296513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d338920-4100-4141-b33a-ee16923e314b None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:37.298140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 589/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Jul 20 09:24:37 2022] GET /identity/v3/domains/f54cfe2306104b0dae52c1c19486516e => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:37.304245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:37.351927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:37.400663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2b6b-ed48-4c28-8024-a59f78a5f1ef None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:37.436281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:37.436938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:37.437218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:37.437494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:37.438202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:37.439659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e2d53-5d74-4886-a790-a0f0e23037d8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:37.465032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 584/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:24:37 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:24:39.522294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdef5-7168-4f88-a010-29e9cf0f2134 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:39.522802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdef5-7168-4f88-a010-29e9cf0f2134 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:39.522903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdef5-7168-4f88-a010-29e9cf0f2134 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:39.524264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 590/1174] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:39.529484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:39.529917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:39.530055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:39.547438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:39.548597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:39.614338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 585/1175] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:39.638623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecdef5-7168-4f88-a010-29e9cf0f2134 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:39.713753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:39.739164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecdef5-7168-4f88-a010-29e9cf0f2134 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:39.772682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:39.773522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:39.773799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:39.774068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] PATH_INFO: `/v3/domains/f54cfe2306104b0dae52c1c19486516e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:39.775219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f54cfe2306104b0dae52c1c19486516e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:39.776662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f344583-0784-48eb-a07c-bc97a4f86b88 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:39.778250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 591/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Jul 20 09:24:39 2022] GET /identity/v3/domains/f54cfe2306104b0dae52c1c19486516e => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:39.783947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:39.813599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:39.880034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738bb6e-158b-4de1-8025-c1228ef5e979 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:39.912263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:39.912931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:39.913213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:39.913493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:39.914203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:39.915677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b0587-0011-4c91-86df-95b8407fd4bf None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:39.935158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 586/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:24:39 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:24:41.993776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ff98-35c2-4dd9-8d77-69b9e2e848f0 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:41.994314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ff98-35c2-4dd9-8d77-69b9e2e848f0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:41.994314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70ff98-35c2-4dd9-8d77-69b9e2e848f0 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:41.995746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 592/1178] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:42.000934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:42.001332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:42.001486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:42.018476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:42.019688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:42.087871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 587/1179] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:42.111208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70ff98-35c2-4dd9-8d77-69b9e2e848f0 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:42.210327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70ff98-35c2-4dd9-8d77-69b9e2e848f0 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:42.243911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:42.244625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:42.244905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:42.245175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:42.254909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:42.262837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:42.264111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:42.265258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 593/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:42.270209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:42.288451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:42.368028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ef1f-6aaa-47c3-8513-2a2a6a7b5de4 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:42.400160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:42.400812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:42.401091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:42.401362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:42.402096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:42.402399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:42.403915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:42.412298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 588/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:42.417993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:42.420891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:42.516658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94871b51-fc8a-4b9d-ab56-d1a50d888b7e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:42.548905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:42.550022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:42.550305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:42.550578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] PATH_INFO: `/v3/users/3347b930514443a9ae12e18875bf6024` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:42.563716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=3347b930514443a9ae12e18875bf6024)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:42.565146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:42.566785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 594/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:42 2022] GET /identity/v3/users/3347b930514443a9ae12e18875bf6024 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:42.572715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:42.650557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:42.669702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3669d07-8f99-493f-a792-a851a24f6cd7 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:42.702244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:42.702926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:42.703212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:42.703517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:42.767324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:42.768685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:42.844625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:42.864501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 589/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:42.871505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:42.934475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:42.973875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb3f1d-7d24-4568-abd4-147cb7b12ca2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:43.007589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:43.008437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:43.008717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:43.008987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] PATH_INFO: `/v3/projects/de3b7718732842c8bdbe93e74141393f` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:43.010155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] RBAC: Authorizing `identity:get_project(project_id=de3b7718732842c8bdbe93e74141393f)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:43.011604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18021ee-2c18-4921-b7c1-68dac32dbbe4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:43.013296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 595/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:42 2022] GET /identity/v3/projects/de3b7718732842c8bdbe93e74141393f => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:43.019472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:43.117665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e987ff8-5c0f-4d00-93e6-dac9d750bb96 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:43.150212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:43.150834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:43.151111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:43.151405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:43.152218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=de3b7718732842c8bdbe93e74141393f, user.id=3347b930514443a9ae12e18875bf6024) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:43.152523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:43.153939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae2928-6931-4394-b7d8-5520474a6f1c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:43.159063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 590/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:43 2022] GET /identity/v3/role_assignments?user.id=3347b930514443a9ae12e18875bf6024&scope.project.id=de3b7718732842c8bdbe93e74141393f&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:45.182057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe2ef-c5f3-407c-8d22-3d524d642ba8 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.182620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe2ef-c5f3-407c-8d22-3d524d642ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.182620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fe2ef-c5f3-407c-8d22-3d524d642ba8 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.184025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 596/1186] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:45.189054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.189447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.189599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.206252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:45.207443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:45.245133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:45.272788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 591/1187] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:45.294652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fe2ef-c5f3-407c-8d22-3d524d642ba8 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:45.393651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fe2ef-c5f3-407c-8d22-3d524d642ba8 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:45.421777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:45.430536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:45.431350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.431631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.431902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.441827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:45.449663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:45.450768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:24:45.451705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 597/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:45.457138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:45.552248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef7b99-ff76-4d64-8f58-bab3e7c044d6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:45.585311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:45.585964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.586246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.586521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.587261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:45.587597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:45.588958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:45.596750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 592/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:45.603198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:45.700147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be6b0-c5ea-4fd3-bf14-8373717ab28a None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:45.732671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:45.733515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.733796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.734069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] PATH_INFO: `/v3/users/3347b930514443a9ae12e18875bf6024` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.735237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] RBAC: Authorizing `identity:get_user(user_id=3347b930514443a9ae12e18875bf6024)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:45.736681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:45.738296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 598/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:45 2022] GET /identity/v3/users/3347b930514443a9ae12e18875bf6024 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:45.744183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:45.778257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:45.841028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab960424-218c-47c9-b391-fc01cf8eca57 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:45.873451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:45.874296 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:45.874579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:45.874850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] PATH_INFO: `/v3/projects/de3b7718732842c8bdbe93e74141393f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:45.876032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] RBAC: Authorizing `identity:get_project(project_id=de3b7718732842c8bdbe93e74141393f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:45.877445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d0ad-9f7e-4736-8020-0cecc6390a84 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:45.879111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 593/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:45 2022] GET /identity/v3/projects/de3b7718732842c8bdbe93e74141393f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:45.885481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:45.940682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:45.983202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3714fe-b4a8-468f-8840-c1743c452e04 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:46.015065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:46.015959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:46.016239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:46.016511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] PATH_INFO: `/v3/projects/de3b7718732842c8bdbe93e74141393f/users/3347b930514443a9ae12e18875bf6024/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:46.018432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] RBAC: Authorizing `identity:create_grant(project_id=de3b7718732842c8bdbe93e74141393f, user_id=3347b930514443a9ae12e18875bf6024, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:46.019890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb64cf1-27df-4039-b9cc-0e5e8c518382 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:46.032525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 599/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:24:45 2022] PUT /identity/v3/projects/de3b7718732842c8bdbe93e74141393f/users/3347b930514443a9ae12e18875bf6024/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:24:48.016748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b054-c351-4840-a40f-2860aee0ffaf None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.017283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b054-c351-4840-a40f-2860aee0ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.017283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4b054-c351-4840-a40f-2860aee0ffaf None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.018662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 594/1193] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:24:48.024399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.024799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.024949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.042012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:24:48.043173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:48.106987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 600/1194] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:24:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:24:48.129559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4b054-c351-4840-a40f-2860aee0ffaf None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:48.229700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4b054-c351-4840-a40f-2860aee0ffaf None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:48.231988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:48.264015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:48.264795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.265076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.265346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.275698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:24:48.283213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:24:48.284344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:24:48.285410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 595/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:24:48.290469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:48.335472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:48.387313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9dc93-503f-426c-8f60-6eb5c4d29cf7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:48.419954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:48.420599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.420879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.421150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.421879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:48.422181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:48.423553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:48.432434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 601/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:24:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:48.438132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:48.536461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd624-a0ea-436b-a52d-c0c81eebbb01 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:48.569042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:48.569936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.570214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.570483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] PATH_INFO: `/v3/users/3347b930514443a9ae12e18875bf6024` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.571675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] RBAC: Authorizing `identity:get_user(user_id=3347b930514443a9ae12e18875bf6024)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:48.573088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:48.574684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 596/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Jul 20 09:24:48 2022] GET /identity/v3/users/3347b930514443a9ae12e18875bf6024 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:48.580468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:48.676064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ce505-a0d4-48b2-8f29-3f97b48104f9 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:48.684936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:48.709740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988527e2-755a-472e-8957-341757d28a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:48.710378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988527e2-755a-472e-8957-341757d28a8e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:48.710660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988527e2-755a-472e-8957-341757d28a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:48.710936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988527e2-755a-472e-8957-341757d28a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:48.775540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988527e2-755a-472e-8957-341757d28a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:48.776970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988527e2-755a-472e-8957-341757d28a8e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:48.870501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 602/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:24:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:24:48.876250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:48.907154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:48.980492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0895d9c-e995-44b1-9598-8e8744e4a1fe None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:49.018959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:49.019841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:49.020117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:49.020385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] PATH_INFO: `/v3/projects/de3b7718732842c8bdbe93e74141393f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:49.021816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=de3b7718732842c8bdbe93e74141393f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:49.023226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad961e98-b1d0-4820-9df9-4046f4700ad9 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:49.024932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 597/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Jul 20 09:24:48 2022] GET /identity/v3/projects/de3b7718732842c8bdbe93e74141393f => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:49.031197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988527e2-755a-472e-8957-341757d28a8e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:49.041963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:49.129117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988527e2-755a-472e-8957-341757d28a8e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:49.162404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:49.163064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:49.163376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:49.163653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:49.164473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=de3b7718732842c8bdbe93e74141393f, user.id=3347b930514443a9ae12e18875bf6024) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:24:49.164784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:49.166211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f6148-ab14-406d-a186-70ea5f7814a8 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:49.171873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 603/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:24:49 2022] GET /identity/v3/role_assignments?user.id=3347b930514443a9ae12e18875bf6024&scope.project.id=de3b7718732842c8bdbe93e74141393f&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:24:59.015326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=50 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:59.169256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=64 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:44.792108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88633a-81c5-4f2e-85e0-620c4974edac None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:44.792731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88633a-81c5-4f2e-85e0-620c4974edac None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:44.792731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88633a-81c5-4f2e-85e0-620c4974edac None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:44.794033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 598/1201] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:25:44.801761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:44.802288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:44.802417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:44.826417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:25:44.827793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:44.912680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 604/1202] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:25:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:25:44.941596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88633a-81c5-4f2e-85e0-620c4974edac None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:45.077863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88633a-81c5-4f2e-85e0-620c4974edac None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:45.115822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:45.116686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:45.116966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:45.117311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:45.130427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:45.131928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:45.156751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:45.158084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:45.159119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:45.159119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:45.159119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:45.159119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:45.159119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 599/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:45.165328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:45.286849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccfbf1-6a06-4fd3-b2e7-3cd79bda7e9e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:45.322503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:45.332413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:45.332999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:45.333323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:45.333553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:45.334272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:45.334595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:45.336073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3ed2e-261d-4c48-8404-bd5393fc99df None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:45.350059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 605/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:25:45.356935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:45.429360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:45.486822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a1073-978e-41bb-8320-00aea98b6aea None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:45.536479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:45.537205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:45.537498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:45.537803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:45.538504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:45.540484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7e651-c3c2-4a9c-a036-f87e0b941cc5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:45.568689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 600/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Jul 20 09:25:45 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:25:47.809248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017014-5393-41bc-af83-a3f9460cd082 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:47.809788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017014-5393-41bc-af83-a3f9460cd082 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:47.809788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017014-5393-41bc-af83-a3f9460cd082 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:47.811470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 606/1206] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:25:47.817081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:47.817607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:47.817607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:47.837164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:25:47.838622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:47.916582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 601/1207] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:25:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:25:47.942110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77017014-5393-41bc-af83-a3f9460cd082 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:48.071661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77017014-5393-41bc-af83-a3f9460cd082 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:48.111511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:48.112310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:48.114760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:48.115049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:48.126822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:25:48.135344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:48.136482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:48.137383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 607/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:25:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:25:48.142697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:48.185808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:48.263201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc9888-7223-46cc-929f-91b413f160a3 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:48.299679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:48.300349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:48.300635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:48.300917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:48.301662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:48.302402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:48.303384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:48.315320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 602/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:25:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:48.318081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:48.376035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:48.444382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906dc9d-dfdd-4a2b-a528-2652fc2e6fdc None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:48.485747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:48.486698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:48.486983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:48.487303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:48.499005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:48.500482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:48.519644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:48.520615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:48.521576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:48.521576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:48.521576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:48.521576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:48.524245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 608/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:25:48.527821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:48.640197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:48.643578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093607fb-4197-463e-ad85-329b1c841d4c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:48.685496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:48.686218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:48.686494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:48.686767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:48.687634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:48.687951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:48.689441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:48.703977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 603/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:48.710305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:48.833980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891424b1-aec9-4faf-a28a-f71a759109c7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:48.836116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:48.874287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:48.875085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:48.875408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:48.875682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:48.884607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:48.886044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:25:48.900870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:48.902283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:48.903357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 609/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:48.909803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:49.034353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92c946-f631-4c64-b22d-43c67b81db2d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:49.077263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:49.078132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:49.078521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:49.078868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:49.079807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:49.080984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:49.081794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:49.107450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 604/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:25:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:49.112521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:49.255835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727873d9-c912-4eee-91ad-5b47044874ea None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:49.296672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:49.297407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:49.297744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:49.298013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:49.340218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:49.390811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:49.392548 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:49.516055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 610/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:25:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:25:49.522036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:49.552443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:49.648452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793081ba-934a-41e7-90b6-40aaf72ece48 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:49.692139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:49.693088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:49.693367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:49.693637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:49.705557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:49.707049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:49.726715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:49.728325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:49.729824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:49.729824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:49.729824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:49.729824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:49.729824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 605/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:49.734685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:49.802240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:49.846838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7bca9-92b5-47d0-a71a-0fa9ccf93b02 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:49.885715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:49.886398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:49.886713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:49.886935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:49.887726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:49.888351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:49.889449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:49.902192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 611/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:49.907811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:49.989032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:50.030377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978c49-ff59-40e3-ad63-c92044bc070c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:50.078481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:50.079268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:50.079579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:50.079851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:50.087427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:50.088876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:25:50.103623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:50.104646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:50.110580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 606/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:50.111124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:50.230003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e9a1-4a89-4d70-952e-251c62bd7bb7 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:50.261260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:50.277687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:50.278358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:50.278642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:50.278920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:50.279741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:50.280080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:50.281497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531c542-9ff9-4ff6-9ea0-c1fbbc429b94 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:50.301481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 612/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:50.305883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:50.443387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae970c4c-d526-476f-8219-d677b7605e27 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:50.485586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:50.491532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:50.492234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:50.492549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:50.492823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:50.493671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=9dddb4f1f2b443049a249f9ab3eeb23f) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:50.493980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:50.495452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daaebc-6224-45d2-9df6-fc3da778b7ef None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:50.501994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 607/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:25:50 2022] GET /identity/v3/role_assignments?user.id=9dddb4f1f2b443049a249f9ab3eeb23f&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:52.699382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869bd7-f632-4433-8e34-7a3eb1cc3073 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:52.699936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869bd7-f632-4433-8e34-7a3eb1cc3073 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:52.699936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65869bd7-f632-4433-8e34-7a3eb1cc3073 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:52.701177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 613/1220] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:25:52.708511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:52.708511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:52.708511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:52.725372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:25:52.726548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:52.797698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 608/1221] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:25:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:25:52.822710 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65869bd7-f632-4433-8e34-7a3eb1cc3073 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:52.931226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65869bd7-f632-4433-8e34-7a3eb1cc3073 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:52.971076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:52.971829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:52.972113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:52.972386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:52.982318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:25:52.990140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:52.991104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:52.992153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 614/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:25:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:52.997780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.101010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79987d5d-92d0-4cdb-b992-fe79b9bcfdd9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:53.125461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:53.136881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:53.137555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:53.137835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:53.138107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:53.138854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:53.139161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:53.140658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:53.149335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 609/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:25:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:53.155068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.202586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:53.266209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b6a1-fdf7-4d45-88c7-a733d5dc09d4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:53.300560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:53.301429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:53.301705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:53.301974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:53.315398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:53.317101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:53.337928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:53.339135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:53.340139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:53.340139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:53.340139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:53.340139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:53.340139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 615/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:53.345477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.453234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb5c03-ca5a-441d-96ba-f9e6d49f7a9f None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:53.486883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:53.487564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:53.487845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:53.488116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:53.488826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:53.489129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:53.490473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:53.502493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 610/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:53.508829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.610932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:53.617904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d2d65-72ba-48d3-bcdd-5b0b07a700bd None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:53.658108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:53.658954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:53.659279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:53.659610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:53.667971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:53.669449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:25:53.685046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:53.686260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:53.687469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 616/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:25:53.692372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.740896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:53.797722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e060a1-f602-40e1-ac93-7160c8aa00ba None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:53.835011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:53.835720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:53.836225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:53.836612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:53.837448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:53.837820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:53.840290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:53.864576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 611/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:25:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:53.869163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:53.981234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbd2b2-70f0-43c0-b827-aa94f414607d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:54.020931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:54.022787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:54.023082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:54.023429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:54.037202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:54.038734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:54.058407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:54.059594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:54.060749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:54.060749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:54.060749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:54.060749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:54.060749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 617/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:54.066351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:54.187887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177603a2-f3d4-49bc-a9fb-54390f269e83 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:54.190066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:54.231586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:54.232581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:54.232940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:54.233446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:54.234349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:54.234734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:54.238286 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:54.252134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 612/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:25:54.257765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:54.268567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:54.369833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd53e8e-fa02-44a5-a0c7-b63d0fdd2eca None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:54.406285 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:54.407637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:54.407931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:54.408208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:54.416344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:54.418171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:25:54.436871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:54.438091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:54.439172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 618/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:25:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:54.447565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:54.553843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295986e-164c-4ee2-a056-cf7061fc760b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:54.590849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:54.591562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:54.591912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:54.592276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:54.593056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:54.593362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:54.594778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1244a9-eb9c-457d-8960-a36f4ae041fd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:54.613095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 613/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:25:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:25:54.614212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:54.684928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:54.723931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a62e6-dc2d-479c-a19d-471ad8b7a7e4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:54.760923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:54.761824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:54.762095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:54.762368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/9dddb4f1f2b443049a249f9ab3eeb23f/roles/09104180e3694124bf32296605caf033` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:54.777583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=9dddb4f1f2b443049a249f9ab3eeb23f, role_id=09104180e3694124bf32296605caf033)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:54.779051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159d22f-3445-4d27-bc07-6cf1ba058ff2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:54.793154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 619/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:25:54 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/9dddb4f1f2b443049a249f9ab3eeb23f/roles/09104180e3694124bf32296605caf033 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:25:57.071816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1a1d-9b6a-4ae5-995d-050317cbc7eb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.072380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1a1d-9b6a-4ae5-995d-050317cbc7eb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.072380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c1a1d-9b6a-4ae5-995d-050317cbc7eb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.073714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 614/1233] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:25:57.079497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.080046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.080210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.098591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:25:57.099807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:57.172274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 620/1234] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:25:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:25:57.198874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1a1d-9b6a-4ae5-995d-050317cbc7eb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:57.231037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:57.328434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c1a1d-9b6a-4ae5-995d-050317cbc7eb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:57.364765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:57.365644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.365974 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.366253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.377789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:25:57.386657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:57.387864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:25:57.388893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:57.393129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 615/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Jul 20 09:25:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:57.394467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:57.443989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:57.498719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c13e7f-c5ae-4368-8534-c6be20b732f7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:57.534557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:57.535359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.535639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.535907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.536642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] RBAC: Adding query filter params (name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:57.537039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:57.538399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:57.547303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 621/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Jul 20 09:25:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:57.553468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:57.649094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:57.668083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c0731-fb82-4231-b85c-900bab141a40 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:57.705639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:57.706428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.706705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.707019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.718474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:57.719966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:57.739720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:57.740915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:57.741846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:57.741846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:57.741846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:57.741846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:57.741846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 616/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:25:57.747907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:57.864225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743c52-5f7a-4f8e-9f9a-354b45469eea None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:57.875478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:57.904416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:57.904989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:57.905273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:57.905554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:57.906271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:57.906576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:57.907983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:57.921229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 622/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:25:57.927266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:58.043636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb79510-a62b-462b-aec4-135bf0a5e022 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:58.082840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:58.083704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:58.083986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:58.084261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:58.087116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:58.093033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:58.094770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:25:58.111856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:58.112935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:25:58.114172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 617/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:58.120979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:58.235965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedcbbe-d3e5-4e67-a176-257c61ebbb3c None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:58.276972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:58.278302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:58.278828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:58.279110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:58.280369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:58.280918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:58.282595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:58.299914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 623/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:25:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:58.305991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:58.417381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73dc1-541e-4a91-9f0a-dc24beb1c097 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:58.455660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:58.456374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:58.456665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:58.457080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:58.537583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:58.545182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:58.546843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:58.660424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 618/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:25:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:25:58.666967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:58.701168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:58.789329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bff7a0-93cb-4d2f-8f7c-0f8fcb9c0ec2 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:58.827375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:58.828142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:58.828510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:58.828745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:58.842750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:58.844611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:25:58.863349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:58.864412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:58.870375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:25:58.870375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:25:58.870375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:25:58.870375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:58.870375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 624/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:25:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:25:58.870702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:58.969832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:58.976578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a71d9-43a3-4d48-9f1c-7011cc998700 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:59.012342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:59.013076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:59.013405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:59.013674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:59.014382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:59.014683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:59.016687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:59.029154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 619/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:59.034986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:59.119906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:59.145935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80a5b8-63f3-4463-b5c7-e87998c48572 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:59.183136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:59.184221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:59.184477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:59.184815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:59.193470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:59.195029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:25:59.211826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:25:59.213192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:25:59.214393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 625/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:25:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:25:59.223059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:59.343475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bcfa3-dae1-4cdc-9734-1180895412af None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:59.387253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:59.387953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:59.388239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:59.388516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:59.389418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:59.389730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:59.391325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30144229-b991-48ee-88d5-5f8e404a3a10 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:59.404558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 620/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:25:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:25:59.410330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:59.530189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc6c71-09e2-4a14-8baa-941b2d1a8a31 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:59.568550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:59.569205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:59.569484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:59.569753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:59.570571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] RBAC: Adding query filter params (role.id=09104180e3694124bf32296605caf033, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=9dddb4f1f2b443049a249f9ab3eeb23f) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:25:59.570930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:59.573248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360666a-220a-4d49-84f3-30d1b6f0216a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:59.580491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 626/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:25:59 2022] GET /identity/v3/role_assignments?user.id=9dddb4f1f2b443049a249f9ab3eeb23f&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=09104180e3694124bf32296605caf033 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:01.773143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c1b2-2d83-40a9-8429-7afca97f7699 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:01.773691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c1b2-2d83-40a9-8429-7afca97f7699 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:01.773691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c1b2-2d83-40a9-8429-7afca97f7699 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:01.775110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 621/1247] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:01.780966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:01.781511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:01.781635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:01.786535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:01.800255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:01.801491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:01.877544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 627/1248] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:01.902553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c1b2-2d83-40a9-8429-7afca97f7699 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:01.922454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:02.029822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c1b2-2d83-40a9-8429-7afca97f7699 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:02.069419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:02.070438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:02.070731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:02.071645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:02.083486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:26:02.093268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:02.094562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:02.100898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 622/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:02.101685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:02.231687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456c8fc-ecef-4517-82e5-27f59b41b61f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:02.267966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:02.274397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:02.275086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:02.275423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:02.276661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:02.276661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:02.276888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:02.278139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:02.294745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 628/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:02.295901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:02.383373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:02.410269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a338-9cba-4705-a41d-c9720d7e7f63 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:02.451469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:02.452337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:02.452603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:02.452871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:02.466757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:02.468258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:02.487530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:02.488766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:02.489777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:02.489777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:02.489777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:02.489777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:02.489777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 623/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:02.495373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:02.622310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcc827-1af7-48aa-94ff-718e0ea51654 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:02.661555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:02.662478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:02.662929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:02.663221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:02.664001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:02.664316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:02.666596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:02.683504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 629/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:02.689933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:02.841150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42398280-ddf1-4fe5-b823-3b99b6b84987 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:02.873744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:02.887183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:02.888193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:02.888479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:02.888759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:02.897853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:02.900011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:26:02.918271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:02.919408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:02.924162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 624/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:02.926377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:02.999098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:03.061613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c132e-e927-45a6-97b4-d8e5f8f404c4 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:03.110119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:03.111219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:03.111534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:03.111842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:03.113235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:03.113552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:03.116047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:03.143991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 630/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:26:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:03.150222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:03.288532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca80db2-164c-45a2-9f8b-0b4a5b66a7a5 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:03.341997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:03.342690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:03.342970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:03.343247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:03.407164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:03.422526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:03.424201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:03.549670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 625/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:26:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:26:03.555644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:03.668305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d690d-120f-4185-9661-388f9b93fe58 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:03.683241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:03.712925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:03.713730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:03.714009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:03.714282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:03.728280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:03.729722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:03.748329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:03.754863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:03.763473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:03.763473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:03.763473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:03.763473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:03.763473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 631/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:26:03.763930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:03.867463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:03.880799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296c5e0-775c-4f19-b8c2-362a9a7fb7fa None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:03.917388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:03.918057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:03.918391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:03.918617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:03.919368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:03.919677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:03.921055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:03.934043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 626/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:03.941062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:04.053529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993e8a2-ad36-43ca-965c-aed9a3a9f421 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:04.092442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:04.093286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:04.093600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:04.093872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:04.101034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:04.102985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:26:04.117796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:04.124519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:04.125695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 632/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:26:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:04.126254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:04.236941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdcb04-857e-48f1-8426-c16f54541c80 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:04.282255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:04.282939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:04.283218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:04.283512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:04.284310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:04.284617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:04.286074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d08d24f-7b45-47ec-87b5-97da3fc5a5da None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:04.299390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 627/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:26:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:04.305693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:04.334752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:04.413396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1f54a-4b4b-4454-8fd6-28bfcea5aa7e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:04.450112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:04.450769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:04.451047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:04.451353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:04.452184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=9dddb4f1f2b443049a249f9ab3eeb23f) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:04.452492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:04.453928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c64d4-2917-469e-a31a-38f67ad419d4 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:04.459757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 633/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:26:04 2022] GET /identity/v3/role_assignments?user.id=9dddb4f1f2b443049a249f9ab3eeb23f&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:06.532442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed022-ff5c-4d48-90f9-fff3ee4f004d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:06.532950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed022-ff5c-4d48-90f9-fff3ee4f004d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:06.533068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bed022-ff5c-4d48-90f9-fff3ee4f004d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:06.534346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 628/1261] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:06.539463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:06.539906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:06.540025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:06.558028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:06.559203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:06.627149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 634/1262] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:06.650854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bed022-ff5c-4d48-90f9-fff3ee4f004d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:06.661419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:06.749337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bed022-ff5c-4d48-90f9-fff3ee4f004d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:06.782392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:06.783218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:06.783530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:06.783805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:06.795071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:26:06.803102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:06.804115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:06.805000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 629/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:06.810932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:06.828834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:06.907718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c830cea-a9e4-4d29-ae4b-906ffed9baa6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:06.940774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:06.941428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:06.941711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:06.941985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:06.942720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:06.943041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:06.944428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:06.953200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 635/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:06.958948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.024304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:07.058130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168918-c8e5-429a-a0f5-44ae9c127c6b None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.091227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.092040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.092323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.092604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.103170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.104631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:07.121263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:07.122395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:07.123353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:07.123353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:07.123353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:07.123353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:07.123353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 630/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:07.128494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.222246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:07.263010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44531da-d3dd-4aeb-9baf-1cc714256230 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.311326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.311909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.312196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.312473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.313192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:07.313502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.314872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.326890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 636/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:07.332706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.430148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3e54-f903-45c5-9ff5-a580912ecbfb None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.449634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:07.464604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.465406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.465685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.465954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.473238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.474675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:26:07.489374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:07.490474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:07.491682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 631/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:07.496338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.592773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1e82e-b875-45e0-8d89-dbf53c3a5599 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.625633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.626277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.626554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.626823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.627672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:07.627979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.629384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.637962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:07.647697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 637/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:26:07.653980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.756312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64fdcc-e2f9-4064-963d-8e31f9d7450d None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.787960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.788734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.789011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.789279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.799947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.801377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:07.819468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:07.820621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:07.821955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:07.821955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:07.821955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:07.821955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:07.821955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 632/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:07.826708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:07.923509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea076e-0da8-4f66-abe4-7505ea550872 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:07.957076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:07.957728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:07.958011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:07.958290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:07.959005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:07.959343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:07.960711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:07.972491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 638/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:07.979029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:08.012068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:08.080334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5e3ee-fe20-46e0-b8c5-81ca488e14ec None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:08.113660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:08.114457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:08.114736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:08.115007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:08.123334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:08.124764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:26:08.140266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:08.141391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:08.142389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 633/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:26:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:08.147426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:08.205216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:08.248626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659559e-c414-4cb0-9e9d-8d5a87db1d42 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:08.281381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:08.282037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:08.282320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:08.282598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:08.283405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:08.283718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:08.285140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d0736-478c-4fb6-bdc4-5f5c6eab7f55 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:08.296716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 639/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:08.302712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:08.393024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:08.402199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40908b9b-2c31-4a57-927e-9df531438ed3 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:08.435109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:08.436044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:08.436324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:08.436596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] PATH_INFO: `/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/9dddb4f1f2b443049a249f9ab3eeb23f/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:08.438526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] RBAC: Authorizing `identity:create_grant(project_id=84bb5538f2344f0aaf64d7e0b22d4a0a, user_id=9dddb4f1f2b443049a249f9ab3eeb23f, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:08.439989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa198f39-e05b-45ca-af58-d0fdf9a4d20f None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:08.453153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 634/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Jul 20 09:26:08 2022] PUT /identity/v3/projects/84bb5538f2344f0aaf64d7e0b22d4a0a/users/9dddb4f1f2b443049a249f9ab3eeb23f/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:26:10.453226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e609-c008-49db-9852-16235941324c None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:10.453776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e609-c008-49db-9852-16235941324c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:10.453776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e609-c008-49db-9852-16235941324c None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:10.455186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 640/1274] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:10.460900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:10.461414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:10.461513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:10.479570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:10.480756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:10.547709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 635/1275] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:10.571304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18e609-c008-49db-9852-16235941324c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:10.673094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18e609-c008-49db-9852-16235941324c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:10.690914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:10.708671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:10.709385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:10.709670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:10.709955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:10.720845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:26:10.728764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:10.729996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:10.731383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 641/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Jul 20 09:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:10.736108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:10.816097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:10.835900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102acd94-48f8-4f42-9036-5473713c94f4 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:10.870241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:10.870904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:10.871181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:10.871489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:10.872231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:10.872530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:10.873881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:10.881893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 636/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Jul 20 09:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:10.888136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:10.988605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b53558-2b71-4967-8008-a74ba7d9c592 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:11.022081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:11.022870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:11.023146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:11.023448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:11.034532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:11.035989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:11.053238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:11.054435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:11.060458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:11.060458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:11.060458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:11.060458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:11.060458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 642/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:11.060874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:11.160008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13347bf4-1df0-4d1c-a666-c04b62a0accc None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:11.193219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:11.193880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:11.194156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:11.194428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:11.195135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:11.195464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:11.197563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:11.209549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 637/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:11.217073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:11.256759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:11.333671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb158-f740-4928-a904-2dcebcd21d04 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:11.367687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:11.369043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:11.369482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:11.369949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:11.377637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:11.379084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:26:11.394108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:11.395264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:11.396443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 643/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:11.401285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:11.405625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:11.501180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4baf8-6505-4c17-a29a-60669e31069c None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:11.533015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:11.533610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:11.533887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:11.534159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:11.534977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:11.535282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:11.536714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:11.553348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 638/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Jul 20 09:26:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:26:11.560110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:11.649019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:11.659928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b2c5c-3782-4990-8268-6696d21a6d36 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:11.694743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:11.695423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:11.695708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:11.696026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:11.761742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:11.763179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:11.856468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:11.866917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 644/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Jul 20 09:26:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:26:11.873007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:11.932713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:11.969884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a751bd8-1a2b-473f-be8b-3d80ae3f3a74 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:12.002279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:12.003057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:12.003363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:12.003640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:12.013929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:12.015377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:26:12.032384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:12.033465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:12.034770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:26:12.034770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:26:12.034770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:26:12.034770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:12.034770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 639/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:12.039095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:12.136433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc209d-3a54-45c9-ae88-39562f9b39cb None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:12.168082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:12.168725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:12.169003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:12.169272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:12.169977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:12.170277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:12.171652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:12.183075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 645/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Jul 20 09:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:12.188725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:12.283470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5fb18-fbc8-4be9-bb0b-130f008007ea None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:12.298040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:12.317916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:12.318694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:12.318973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:12.319242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:12.326287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:12.327741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:26:12.341842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:12.342897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: ERROR keystone.server.flask.application Jul 20 09:26:12.344102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 640/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Jul 20 09:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:12.348885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:12.397315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:12.447954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb23012-825d-4974-a3da-b0dab68c932d None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:12.480919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:12.481570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:12.481852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:12.482127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:12.482906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:12.483219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:12.484674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f13e-42bb-46f7-be76-fab280a0793a None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:12.496854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 646/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Jul 20 09:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:12.502851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:12.598925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9be5b9-6f41-4768-a6d8-3f162319da23 None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:12.630863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:12.631528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:12.631811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:12.632082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:12.632894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] RBAC: Adding query filter params (role.id=743f7cb41c634460b2981782f093ad19, scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a, user.id=9dddb4f1f2b443049a249f9ab3eeb23f) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:12.633202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:12.634681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c6eb15-df14-4d2a-91ff-b77a7fafc587 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:12.640356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 641/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Jul 20 09:26:12 2022] GET /identity/v3/role_assignments?user.id=9dddb4f1f2b443049a249f9ab3eeb23f&scope.project.id=84bb5538f2344f0aaf64d7e0b22d4a0a&role.id=743f7cb41c634460b2981782f093ad19 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:14.669538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c973d-4bc4-464a-9c8f-ba4298de9462 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:14.670081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c973d-4bc4-464a-9c8f-ba4298de9462 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:14.670081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c973d-4bc4-464a-9c8f-ba4298de9462 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:14.671514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 647/1288] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:14.676779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:14.677215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:14.677317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:14.693832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:14.694928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:14.723902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:14.760792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 642/1289] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:14.783178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c973d-4bc4-464a-9c8f-ba4298de9462 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:14.881626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c973d-4bc4-464a-9c8f-ba4298de9462 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:14.892714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:14.915440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312154b7-3280-4269-a951-46163b54a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:14.916417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312154b7-3280-4269-a951-46163b54a01e None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:14.916698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312154b7-3280-4269-a951-46163b54a01e None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:14.916969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312154b7-3280-4269-a951-46163b54a01e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:14.917611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312154b7-3280-4269-a951-46163b54a01e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:14.918971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312154b7-3280-4269-a951-46163b54a01e None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-312154b7-3280-4269-a951-46163b54a01e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:14.930570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:14.932685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:14.933888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 648/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:26:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:14.938484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:15.034493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643152e6-5545-40c1-8921-fe70e3d5249e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:15.067466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:15.068116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:15.068400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:15.068677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:15.069372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:15.069678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:15.071032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4321e8-9f88-4408-bbbf-ad84085d44e8 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:15.075923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 643/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jul 20 09:26:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:15.081554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312154b7-3280-4269-a951-46163b54a01e None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:15.179066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312154b7-3280-4269-a951-46163b54a01e None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:15.214291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:15.214979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:15.215367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:15.215638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] PATH_INFO: `/v3/services` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:15.216454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:15.216800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:15.218342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d78f04-f545-4d92-a01d-891934981dc0 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:15.228890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 649/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jul 20 09:26:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:17.257686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082b0e-751b-4dba-b5d0-19a6bcb2dad5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:17.258250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082b0e-751b-4dba-b5d0-19a6bcb2dad5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:17.258250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082b0e-751b-4dba-b5d0-19a6bcb2dad5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:17.260292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 644/1293] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:17.265428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94319-c94d-466e-99a6-ab441cbf513c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:17.265806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94319-c94d-466e-99a6-ab441cbf513c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:17.265984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b94319-c94d-466e-99a6-ab441cbf513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:17.282594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-27b94319-c94d-466e-99a6-ab441cbf513c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:17.283824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-27b94319-c94d-466e-99a6-ab441cbf513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:17.313177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:17.349321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 650/1294] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:17.373084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e082b0e-751b-4dba-b5d0-19a6bcb2dad5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:17.403907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:17.474207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e082b0e-751b-4dba-b5d0-19a6bcb2dad5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:17.506796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:17.507476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:17.507809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:17.508028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:17.508740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:17.510088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb16ad8-61ae-45a7-b8d5-dfe5ef593718 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:17.522651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 645/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Jul 20 09:26:17 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:26:19.579839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb834e8c-3d64-4d12-881e-a82713e89889 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:19.580362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb834e8c-3d64-4d12-881e-a82713e89889 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:19.580362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb834e8c-3d64-4d12-881e-a82713e89889 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:19.581803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 651/1296] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:19.586842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:19.587342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:19.587440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:19.604013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:19.605191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:19.677195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 646/1297] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:19.700774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb834e8c-3d64-4d12-881e-a82713e89889 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:19.801218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb834e8c-3d64-4d12-881e-a82713e89889 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:19.835958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:19.836754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:19.837035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:19.837314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:19.838296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:19.839600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:19.851586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:19.852663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:19.854513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 652/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:26:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:19.859973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:19.878792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:19.998135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93925f44-6c65-4f7b-95b0-8bf465ceaf01 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:20.031695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:20.032345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:20.032624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:20.032893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:20.033581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] RBAC: Adding query filter params (name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:20.033880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:20.035233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:20.040246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 647/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jul 20 09:26:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:20.046889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:20.078629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:20.147072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3384a-ab2b-431f-a6de-30175e4afe6f None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:20.181430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:20.182189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:20.182474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:20.183017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:20.183836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] RBAC: Adding query filter params (interface=public, service_id=d6eab2d55e434f7fa936dc635cc2efe0, region_id=RegionOne) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:20.184147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:20.185529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593543e-a8fa-47d1-a324-838b80568046 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:20.190589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 653/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Jul 20 09:26:20 2022] GET /identity/v3/endpoints?service_id=d6eab2d55e434f7fa936dc635cc2efe0&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:20.196294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:20.280156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:20.298975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c050838-6078-446b-896b-2b51ddb396de None admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:20.332656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:20.333325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:20.333606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:20.333880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:20.334551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:20.335935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba67f74-58f5-48e5-84d3-294449c26761 None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:20.343397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 648/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Jul 20 09:26:20 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:26:22.426493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb70-be2d-43cf-855f-77c077719b39 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:22.427030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb70-be2d-43cf-855f-77c077719b39 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:22.427030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71cb70-be2d-43cf-855f-77c077719b39 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:22.428449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 654/1302] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:26:22.434639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:22.435168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:22.435168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:22.453507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:26:22.454720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:22.520323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 649/1303] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:26:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:26:22.542750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71cb70-be2d-43cf-855f-77c077719b39 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:22.636300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:22.643753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71cb70-be2d-43cf-855f-77c077719b39 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:22.679151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eefad-2423-4316-9f79-41e113897352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:22.679886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eefad-2423-4316-9f79-41e113897352 None admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:22.680164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eefad-2423-4316-9f79-41e113897352 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:22.680431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eefad-2423-4316-9f79-41e113897352 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:22.681073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eefad-2423-4316-9f79-41e113897352 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:22.682486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eefad-2423-4316-9f79-41e113897352 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application [None req-154eefad-2423-4316-9f79-41e113897352 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:26:22.694251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:26:22.695363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application with Lock( Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: ERROR keystone.server.flask.application Jul 20 09:26:22.696388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 655/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Jul 20 09:26:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:26:22.702153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:22.779027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:22.801645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674521-f95e-42a7-b191-16f91111ba20 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:22.834355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:22.835005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:22.835369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:22.835650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] PATH_INFO: `/v3/services` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:22.836343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:26:22.836645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:22.838571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ab724-d0aa-433d-8f9e-6fa9b110c3dd None admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:22.844235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 650/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Jul 20 09:26:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:26:22.850127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eefad-2423-4316-9f79-41e113897352 None admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:22.953939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eefad-2423-4316-9f79-41e113897352 None admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:23.000812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:23.001474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:23.001757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:23.002031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:23.002788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:23.004187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c2c7e-7c4b-43c4-8347-fbc70712b792 None admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:23.022973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 656/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Jul 20 09:26:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:26:32.841220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=30 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:33.018594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:25.937372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225f1e-a584-472b-bfd7-f87f781acad0 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:25.937841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225f1e-a584-472b-bfd7-f87f781acad0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:25.937944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc225f1e-a584-472b-bfd7-f87f781acad0 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:25.943674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 651/1307] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:25.945172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f88939-483b-466f-b0c6-a06693a8021b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:25.945592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f88939-483b-466f-b0c6-a06693a8021b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:25.945769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f88939-483b-466f-b0c6-a06693a8021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:26.104221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-94f88939-483b-466f-b0c6-a06693a8021b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:26.108324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-94f88939-483b-466f-b0c6-a06693a8021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:26.296704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 657/1308] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:26.347977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:26.348266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:26.348693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:26.350286 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 652/1309] 10.222.0.18 () {54 vars in 965 bytes} [Wed Jul 20 09:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:26.356346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71858df-7688-4698-8201-fa83161afb12 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:26.356637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71858df-7688-4698-8201-fa83161afb12 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:26.356939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71858df-7688-4698-8201-fa83161afb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:26.441090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f71858df-7688-4698-8201-fa83161afb12 None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:26.441090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f71858df-7688-4698-8201-fa83161afb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:26.462608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:26.524417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 658/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Jul 20 09:27:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:27:26.534720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:26.536655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:26.659953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdafaa-437e-40b5-b3d8-f026cbdef5fb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:26.736723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:26.737379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:26.737658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:26.737928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:26.739659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:26.797043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:26.854461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:26.855938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa2838c-2ca0-4092-8521-226ee67ba573 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:26.876028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 653/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:29.219517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:29.220077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:29.220077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:29.221445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 659/1312] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:29.226822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61233-3634-41ce-a171-c84c2ffd4548 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:29.227243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61233-3634-41ce-a171-c84c2ffd4548 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:29.227392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61233-3634-41ce-a171-c84c2ffd4548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:29.307419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c1e61233-3634-41ce-a171-c84c2ffd4548 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:29.308632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c1e61233-3634-41ce-a171-c84c2ffd4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:29.414126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 654/1313] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:29.454929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:29.456522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:29.559969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b12921-85ca-4373-abb1-b7c57eda32be None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:29.631528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:29.634369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:29.635304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:29.635757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:29.636089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:29.638009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:29.753299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:29.756116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d2380-94b3-43fa-b242-61909b7447f7 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:29.774691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 660/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:32.123448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:32.124148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:32.124457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:32.125848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 655/1315] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:32.131174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537903-eee7-418c-b6c6-5f051b066acb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:32.131650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537903-eee7-418c-b6c6-5f051b066acb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:32.131828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537903-eee7-418c-b6c6-5f051b066acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:32.203646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4f537903-eee7-418c-b6c6-5f051b066acb None None] MFA Rules not processed for user `d3f391bb67354ffe82a26d3d321c1224`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:32.204845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4f537903-eee7-418c-b6c6-5f051b066acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:32.255902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:32.300832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 661/1316] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:27:32.341509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:32.343246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:32.352977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:32.452658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418899-b743-4d39-b586-2c9bb7759dc5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:32.525074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:32.525710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:32.525990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:32.526264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:32.527908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:32.588670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:32.634997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:32.636370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afad301-6a89-4898-a14d-f717d9185e03 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:32.653815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 656/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:35.208880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:35.209422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:35.209422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:35.210800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 662/1318] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:35.216035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cd14e-5f1e-4926-b9e3-1707c5ba96b3 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:35.216557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cd14e-5f1e-4926-b9e3-1707c5ba96b3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:35.216650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cd14e-5f1e-4926-b9e3-1707c5ba96b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:35.281975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5f6cd14e-5f1e-4926-b9e3-1707c5ba96b3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:35.283189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5f6cd14e-5f1e-4926-b9e3-1707c5ba96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:35.376960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 657/1319] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:35.415839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:35.417392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:35.520353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778fae6-6287-4374-a2f5-7473e3f50f08 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:35.591645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:35.593479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:35.593763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:35.594037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:35.595725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:35.602924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:35.700598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:35.702749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b7b3b-0cdc-4df1-97d3-b82e94b1747c service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:35.720484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 663/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:38.409427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:38.409972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:38.409972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:38.411531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 658/1321] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:38.416707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368604-5df4-4fd7-b54b-2256b9ff022c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:38.417144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368604-5df4-4fd7-b54b-2256b9ff022c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:38.417243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b368604-5df4-4fd7-b54b-2256b9ff022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:38.495371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4b368604-5df4-4fd7-b54b-2256b9ff022c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:38.496593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4b368604-5df4-4fd7-b54b-2256b9ff022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:38.557289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:38.600925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 664/1322] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:38.639031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:38.640721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:38.661583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:38.751068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5a22-f1fc-4a84-b3a9-4df33a3c4ebe None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:38.825675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:38.826327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:38.826638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:38.826915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:38.832949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:38.905895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:38.946039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:38.947429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8450bb-db12-4734-90cc-120f832f1e5a service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:38.968130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 659/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:41.093198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:41.093715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:41.093715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:41.095041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 665/1324] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:41.100336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11a39-9e55-434b-9fa4-2f79b3d37fc0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:41.100754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11a39-9e55-434b-9fa4-2f79b3d37fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:41.100896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb11a39-9e55-434b-9fa4-2f79b3d37fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:41.166374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0cb11a39-9e55-434b-9fa4-2f79b3d37fc0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:41.167635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0cb11a39-9e55-434b-9fa4-2f79b3d37fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:41.263619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 660/1325] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:41.300260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:41.302161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:41.416818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c048cef-89d7-4cd0-82b8-02f5aae52a06 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:41.501790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:41.502558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:41.502755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:41.502975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:41.504720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:41.522957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:41.615218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:41.616703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275c86e-8e13-4f45-af6e-b38616273db4 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:41.637245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 666/1326] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:43.749584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed8e8-cabe-43a9-ae09-d8afa034c9d6 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:43.750068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed8e8-cabe-43a9-ae09-d8afa034c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:43.750174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ed8e8-cabe-43a9-ae09-d8afa034c9d6 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:43.751578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 661/1327] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:43.757020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0e0e4-78f1-4e15-86e2-4de96bed21ea None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:43.757476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0e0e4-78f1-4e15-86e2-4de96bed21ea None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:43.757576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0e0e4-78f1-4e15-86e2-4de96bed21ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:43.825087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-04d0e0e4-78f1-4e15-86e2-4de96bed21ea None None] MFA Rules not processed for user `d3f391bb67354ffe82a26d3d321c1224`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:43.826295 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-04d0e0e4-78f1-4e15-86e2-4de96bed21ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:43.895818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:43.929545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 667/1328] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:43.980408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:43.980691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:43.980961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:43.982341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 662/1329] 10.222.0.18 () {54 vars in 965 bytes} [Wed Jul 20 09:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:43.988820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49791d-bb7b-43f2-a446-606a467db18b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:43.989053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49791d-bb7b-43f2-a446-606a467db18b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:43.989334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49791d-bb7b-43f2-a446-606a467db18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:44.063916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-aa49791d-bb7b-43f2-a446-606a467db18b None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:44.065085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-aa49791d-bb7b-43f2-a446-606a467db18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:44.142203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 668/1330] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Jul 20 09:27:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:27:44.151045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:44.152741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:44.183756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:44.262733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5ba9f-cca6-4af1-8035-9d903a649b3e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:44.350103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:44.351259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:44.351585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:44.351862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:44.354838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:44.432558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:44.465949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:44.467460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f529e3-d9be-4fa0-8197-6d2efb11fe7b service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:44.485848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 663/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:47.192622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:47.193193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:47.193193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:47.194537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 669/1332] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:47.199864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e97c7-d0db-454f-8193-ea72b4b37752 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:47.200317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e97c7-d0db-454f-8193-ea72b4b37752 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:47.200425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e97c7-d0db-454f-8193-ea72b4b37752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:47.267111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-154e97c7-d0db-454f-8193-ea72b4b37752 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:47.268351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-154e97c7-d0db-454f-8193-ea72b4b37752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:47.365060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 664/1333] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:47.401290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:47.402814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:47.474100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:47.507372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d130-eefc-4d1a-9a01-adf04379ad39 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:47.578620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:47.579279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:47.579587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:47.579861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:47.581501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:47.686740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:47.688264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af1fd4-1c62-4419-b63a-4e7ce84b87cc service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:47.698340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:47.707353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 670/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:49.788018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:49.788513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:49.788627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:49.789963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 665/1335] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:49.794969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea88bac-b949-4f92-8430-19e8f6509c45 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:49.795402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea88bac-b949-4f92-8430-19e8f6509c45 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:49.795557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea88bac-b949-4f92-8430-19e8f6509c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:49.861639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1ea88bac-b949-4f92-8430-19e8f6509c45 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:49.862835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1ea88bac-b949-4f92-8430-19e8f6509c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:49.960818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 671/1336] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:50.002838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:50.004407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:50.043145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:50.106740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018dd87-a456-4733-8932-39d1fafc0542 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:50.178054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:50.178694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:50.178980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:50.179253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:50.180911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:50.261846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:50.288114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:50.289526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f0add-5136-4290-b595-dfe564f6ff19 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:50.307704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 666/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:52.628615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:52.629168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:52.629168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:52.630526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 672/1338] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:52.635779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae90ff-9bbf-488f-abbf-fd556718703d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:52.636170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae90ff-9bbf-488f-abbf-fd556718703d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:52.636319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae90ff-9bbf-488f-abbf-fd556718703d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:52.699821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2fae90ff-9bbf-488f-abbf-fd556718703d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:52.700925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2fae90ff-9bbf-488f-abbf-fd556718703d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:52.795164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 667/1339] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:52.832868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:52.834421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:52.909246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:52.937428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c476f-2eda-4ffe-a7b2-b6e82dfd3651 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:53.013439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:53.014081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:53.014362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:53.014643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:53.016599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:53.124058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:53.125585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead165ef-cdf4-432e-a1b2-195a9f09d518 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:53.143545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 673/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:57.130672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122191-50e8-41f4-818f-821b9046a83a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:57.131210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122191-50e8-41f4-818f-821b9046a83a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:57.131210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52122191-50e8-41f4-818f-821b9046a83a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:57.132611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 668/1341] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:27:57.138148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436ac9-6022-4a7f-a17b-8a33453b4046 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:57.138572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436ac9-6022-4a7f-a17b-8a33453b4046 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:57.138831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5436ac9-6022-4a7f-a17b-8a33453b4046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:57.143914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:57.207205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f5436ac9-6022-4a7f-a17b-8a33453b4046 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:57.208453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f5436ac9-6022-4a7f-a17b-8a33453b4046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:57.308466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 674/1342] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:57.347550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52122191-50e8-41f4-818f-821b9046a83a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:57.349072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-52122191-50e8-41f4-818f-821b9046a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:57.397749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:57.455417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52122191-50e8-41f4-818f-821b9046a83a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:57.528365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:57.529022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:57.529308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:57.529584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:57.531409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:57.624869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:57.641325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:57.642768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25811c3-8d78-4622-bc54-b54f8a077d97 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:57.661158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 669/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:28:00.372512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:00.373081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:00.373081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:00.374787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 675/1344] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:00.380982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd79e7-79f1-4842-8a74-cd3174e1d8a6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:00.381498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd79e7-79f1-4842-8a74-cd3174e1d8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:00.381577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd79e7-79f1-4842-8a74-cd3174e1d8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:00.457463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c4dd79e7-79f1-4842-8a74-cd3174e1d8a6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:00.458706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4dd79e7-79f1-4842-8a74-cd3174e1d8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:00.561070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 670/1345] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:00.599562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:00.601104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:00.690449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:00.708656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933dbd-c82f-4c89-ad75-1f95f2b64f5b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:00.788260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:00.788835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:00.789121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:00.789400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:00.791040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:00.900664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:00.902162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb291765-166d-419c-a555-03342813ea29 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:00.919917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 676/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:04.924472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:04.925011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:04.925011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:04.926422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 671/1347] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:04.931951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da835-513c-4edb-9855-b9c677f980d1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:04.932370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da835-513c-4edb-9855-b9c677f980d1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:04.932483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4da835-513c-4edb-9855-b9c677f980d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:04.948019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:04.998354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4c4da835-513c-4edb-9855-b9c677f980d1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:04.999603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4c4da835-513c-4edb-9855-b9c677f980d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:05.100758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 677/1348] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:05.140249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:05.141878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:05.222223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:05.275837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d897319-00ce-4593-8ff1-69f710ff19ca None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:05.360670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:05.361262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:05.361551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:05.361967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:05.363960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:05.484627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:05.489938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:05.491419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40f205-bd8f-4abb-a2c4-846ce6d8b199 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:05.511668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 672/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:10.659491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5f22-6fe6-48f0-99d1-7fd6de15f4c6 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:10.660554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5f22-6fe6-48f0-99d1-7fd6de15f4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:10.661034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b5f22-6fe6-48f0-99d1-7fd6de15f4c6 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:10.662875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 678/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:28:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:10.676675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5425d8-a336-4302-a507-cb159410ff40 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:10.677125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5425d8-a336-4302-a507-cb159410ff40 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:10.677206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5425d8-a336-4302-a507-cb159410ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:10.799227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7c5425d8-a336-4302-a507-cb159410ff40 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:10.800667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7c5425d8-a336-4302-a507-cb159410ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:10.923338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 673/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:28:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:15.015041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d04b-2a62-4f4a-a6d5-93ada7141776 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:15.016338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d04b-2a62-4f4a-a6d5-93ada7141776 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:15.016939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d04b-2a62-4f4a-a6d5-93ada7141776 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:15.018618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 679/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:28:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:15.024093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5217d-2992-4342-b94e-bdc706781de7 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:15.024570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5217d-2992-4342-b94e-bdc706781de7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:15.024662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5217d-2992-4342-b94e-bdc706781de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:15.082142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:15.098847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=59 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:15.108071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-65a5217d-2992-4342-b94e-bdc706781de7 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:15.109298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-65a5217d-2992-4342-b94e-bdc706781de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:15.221611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 674/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:28:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:16.545830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e73cf-70d5-4b7a-97a4-a4a54d6c36f4 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:16.546371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e73cf-70d5-4b7a-97a4-a4a54d6c36f4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:16.546371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e73cf-70d5-4b7a-97a4-a4a54d6c36f4 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:16.548391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 680/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:16.553876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5608bf8-080e-4210-a862-d3b08b749837 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:16.554373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5608bf8-080e-4210-a862-d3b08b749837 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:16.554466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5608bf8-080e-4210-a862-d3b08b749837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:16.630985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d5608bf8-080e-4210-a862-d3b08b749837 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:16.632225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d5608bf8-080e-4210-a862-d3b08b749837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:16.676243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:16.742925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 675/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:28:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:20.271256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6194598-76c2-4255-91d3-e81ec49c59e4 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:20.271790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6194598-76c2-4255-91d3-e81ec49c59e4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:20.271790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6194598-76c2-4255-91d3-e81ec49c59e4 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:20.273109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 681/1356] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:20.278613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2823-bacd-4aa4-9999-5cddbee42948 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:20.279500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2823-bacd-4aa4-9999-5cddbee42948 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:20.279500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2823-bacd-4aa4-9999-5cddbee42948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:20.358401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8bab2823-bacd-4aa4-9999-5cddbee42948 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:20.359989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8bab2823-bacd-4aa4-9999-5cddbee42948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:20.475336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 676/1357] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:20.529894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:20.530731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:20.531399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:20.533889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 682/1358] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jul 20 09:28:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:20.543359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae234-5dcc-4e53-a8f8-a5e63eb16d41 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:20.545082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae234-5dcc-4e53-a8f8-a5e63eb16d41 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:20.545621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae234-5dcc-4e53-a8f8-a5e63eb16d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:20.562858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:20.634490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fd2ae234-5dcc-4e53-a8f8-a5e63eb16d41 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:20.635759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fd2ae234-5dcc-4e53-a8f8-a5e63eb16d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:20.763621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 677/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jul 20 09:28:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:20.772016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:20.773562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:20.930509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a9086-0258-4366-b27f-0727a3089d5d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:21.010548 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:21.053098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:21.053671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:21.053952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:21.054226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:21.055879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:21.148502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:21.150109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045eef08-dfda-4103-9c5a-9a00a206452a service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:21.170920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 683/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:28:23.424465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:23.424465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:23.424465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:23.426216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 678/1361] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:23.431493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae106e6-6223-41be-9690-94cc045ac61d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:23.431911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae106e6-6223-41be-9690-94cc045ac61d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:23.431911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae106e6-6223-41be-9690-94cc045ac61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:23.507128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2ae106e6-6223-41be-9690-94cc045ac61d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:23.508622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2ae106e6-6223-41be-9690-94cc045ac61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:23.548880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:23.615532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 684/1362] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:23.663950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:23.665572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:23.739057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:23.838179 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eefd23-ff1f-4914-9f9d-29467ae92e7f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:23.993226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:23.993885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:23.994171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:23.994448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:23.996115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:24.020817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:24.096969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:24.098571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e4516-8aee-45cf-8139-4117808fe5d0 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:24.117832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 679/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:26.508755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baa2a9-8d7f-4f37-8384-3ba35e23ad05 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:26.509289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baa2a9-8d7f-4f37-8384-3ba35e23ad05 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:26.509289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baa2a9-8d7f-4f37-8384-3ba35e23ad05 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:26.510686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 685/1364] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:26.516279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25556-dc75-4245-9911-286c039ee522 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:26.516732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25556-dc75-4245-9911-286c039ee522 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:26.517351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab25556-dc75-4245-9911-286c039ee522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:26.627124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fab25556-dc75-4245-9911-286c039ee522 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:26.627124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fab25556-dc75-4245-9911-286c039ee522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:26.732632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:26.740802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 680/1365] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:26.782657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:26.783404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:26.783530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:26.785469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 686/1366] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jul 20 09:28:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:26.790785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d192e-1340-446a-be61-c7fcdc64063d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:26.791149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d192e-1340-446a-be61-c7fcdc64063d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:26.791518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d192e-1340-446a-be61-c7fcdc64063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:26.871005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-4d6d192e-1340-446a-be61-c7fcdc64063d None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:26.872335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4d6d192e-1340-446a-be61-c7fcdc64063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:27.005030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 681/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jul 20 09:28:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:27.012698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:27.014341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:27.180156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8db556-cf4d-496c-984c-899906980b63 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:27.203118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:27.305936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:27.306765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:27.307169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:27.307562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:27.309316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:27.408256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:27.410102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e201dc-67b9-432c-88a2-d72fea5a5698 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:27.429915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 687/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:28:28.480906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d875c-cd66-4a36-9ea6-0636dab4fefc None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:28.481433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d875c-cd66-4a36-9ea6-0636dab4fefc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:28.481546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d875c-cd66-4a36-9ea6-0636dab4fefc None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:28.482870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 682/1369] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:28.488398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953984-4974-48e7-95f8-f3be5bbb3b0c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:28.488846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953984-4974-48e7-95f8-f3be5bbb3b0c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:28.488929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b953984-4974-48e7-95f8-f3be5bbb3b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:28.522701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:28.560194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9b953984-4974-48e7-95f8-f3be5bbb3b0c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:28.561540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9b953984-4974-48e7-95f8-f3be5bbb3b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:28.672797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 688/1370] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:29.021584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:29.021873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:29.022153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:29.024758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 683/1371] 10.222.0.18 () {54 vars in 964 bytes} [Wed Jul 20 09:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:29.029150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff3a2-7f45-43fe-a734-cfad23aca150 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:29.029442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff3a2-7f45-43fe-a734-cfad23aca150 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:29.029722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ff3a2-7f45-43fe-a734-cfad23aca150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:29.119883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c56ff3a2-7f45-43fe-a734-cfad23aca150 None None] MFA Rules not processed for user `267cd2f0b4d442ee9ab2ee668394472c`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:29.121104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c56ff3a2-7f45-43fe-a734-cfad23aca150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:29.175411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:29.208890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 689/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Jul 20 09:28:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:29.216869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:29.218484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:29.274054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:29.332675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac34401-a187-4c29-b1d2-62df2d8da08a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:29.409420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:29.410083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:29.410366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:29.410639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:29.412327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:29.523912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:29.539277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:29.540662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c8ea77-a3d2-458d-9ed8-b4d0ee95f2a1 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:29.568662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 684/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:29.589366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:29.589830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:29.590068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:29.594970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 690/1374] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:29.598229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e450ae-51d9-4346-bb58-fec840bed632 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:29.599035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e450ae-51d9-4346-bb58-fec840bed632 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:29.599766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e450ae-51d9-4346-bb58-fec840bed632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:29.678444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-76e450ae-51d9-4346-bb58-fec840bed632 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:29.679764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-76e450ae-51d9-4346-bb58-fec840bed632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:29.794891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 685/1375] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:29.834477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:29.835300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:30.006032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b14f-e2bf-4534-a3e4-7b42d7b540d9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:30.057489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:30.121190 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:30.121841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:30.122127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:30.122407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:30.124105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:30.214721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:30.216377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75b77-1e6a-4eef-a458-4b82316eef5c service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:30.237673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 691/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:32.318376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:32.318917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:32.318917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:32.320714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 686/1377] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:32.326028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748e2a-b389-471d-a776-516ebe959038 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:32.326441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748e2a-b389-471d-a776-516ebe959038 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:32.326583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748e2a-b389-471d-a776-516ebe959038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:32.390673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-97748e2a-b389-471d-a776-516ebe959038 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:32.391849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-97748e2a-b389-471d-a776-516ebe959038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:32.398507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:32.508434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 692/1378] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:32.553803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:32.555361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:32.634297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:32.722026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c20c37-398d-42be-920e-8c501e1959b1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:32.834782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:32.835566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:32.835857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:32.836131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:32.837774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:32.861088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:32.934654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:32.936289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02e0c-70ef-4097-8ff0-0533eb4dc0a9 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:32.961998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 687/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:35.072954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:35.073481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:35.073481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:35.074964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 693/1380] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:35.080175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790acf-3058-4a67-b3e6-5eb1fa3defae None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:35.080585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790acf-3058-4a67-b3e6-5eb1fa3defae None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:35.080870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19790acf-3058-4a67-b3e6-5eb1fa3defae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:35.155489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-19790acf-3058-4a67-b3e6-5eb1fa3defae None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:35.156822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-19790acf-3058-4a67-b3e6-5eb1fa3defae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:35.261347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:35.268471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 688/1381] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:35.312141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:35.313857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:35.471100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:35.482529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac750e1-bbcd-4b4e-b613-4c3beaa64716 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:35.609191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:35.609845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:35.610127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:35.610400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:35.612762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:35.714800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:35.716374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdefeda2-c21c-4650-be46-642f997b484f service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:35.727672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:35.738611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 694/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:37.831501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:37.832183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:37.832183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:37.833440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 689/1383] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:37.838906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879aaf-8be2-477b-afdc-a96679a4525e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:37.839517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879aaf-8be2-477b-afdc-a96679a4525e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:37.839517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1879aaf-8be2-477b-afdc-a96679a4525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:37.910737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c1879aaf-8be2-477b-afdc-a96679a4525e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:37.912389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c1879aaf-8be2-477b-afdc-a96679a4525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:38.030907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 695/1384] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:38.072771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:38.074308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:38.226824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ddb0-e81b-4d99-ae35-04a95b5a7f8d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:38.302336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:38.350471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:38.351124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:38.351441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:38.351717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:38.353390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:38.445195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:38.447084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f25f0-3dd8-4ce9-a358-8d5fe7b91ed9 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:38.465765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 690/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:40.812323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:40.812868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:40.813033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:40.816069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 696/1386] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:40.821543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b4a2-370e-44e4-8a98-0a022bb2545f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:40.821992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b4a2-370e-44e4-8a98-0a022bb2545f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:40.822211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70b4a2-370e-44e4-8a98-0a022bb2545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:40.907448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fe70b4a2-370e-44e4-8a98-0a022bb2545f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:40.910023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fe70b4a2-370e-44e4-8a98-0a022bb2545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:40.960923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:41.025957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 691/1387] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:41.072183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:41.073630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:41.152172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:41.227739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28a4fe-f0e5-4e36-917b-612fb336054e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:41.358507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:41.359073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:41.359390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:41.359669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:41.361327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:41.404156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:41.473120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:41.474749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831540d-6390-4aee-abcc-afc150691871 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:41.493241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 697/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:43.772606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:43.773087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:43.773087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:43.774471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 692/1389] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:43.780346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd071-ba60-4954-b30c-2813f2e89c53 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:43.780938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd071-ba60-4954-b30c-2813f2e89c53 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:43.780938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd071-ba60-4954-b30c-2813f2e89c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:43.848313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8e0fd071-ba60-4954-b30c-2813f2e89c53 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:43.849532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8e0fd071-ba60-4954-b30c-2813f2e89c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:43.945174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 698/1390] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:43.990087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:43.991848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:44.137156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cc3df-28d0-4feb-9bfe-15d93d204de2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:44.154019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:44.249675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:44.250323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:44.250596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:44.250866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:44.253053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:44.341079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:44.342666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f730ee-1dfe-4738-9862-3dc2f36156a8 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:44.360646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 693/1391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:44.721426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d2eb-33d5-4cf9-a458-d638a01c4275 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:44.721634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d2eb-33d5-4cf9-a458-d638a01c4275 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:44.721883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d2eb-33d5-4cf9-a458-d638a01c4275 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:44.723302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 699/1392] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:44.728600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16d75-d96f-453c-b8bf-e56e8a4660ad None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:44.728889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16d75-d96f-453c-b8bf-e56e8a4660ad None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:44.729199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c16d75-d96f-453c-b8bf-e56e8a4660ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:44.755512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:44.796048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b2c16d75-d96f-453c-b8bf-e56e8a4660ad None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:44.797242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b2c16d75-d96f-453c-b8bf-e56e8a4660ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:44.892308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 694/1393] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:44.948573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:44.949058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:44.949166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:44.950461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 700/1394] 10.222.0.18 () {54 vars in 966 bytes} [Wed Jul 20 09:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:44.955429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2828d5-6d14-4d15-85b9-a3411903cb85 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:44.956273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2828d5-6d14-4d15-85b9-a3411903cb85 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:44.956602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2828d5-6d14-4d15-85b9-a3411903cb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:45.030590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-cc2828d5-6d14-4d15-85b9-a3411903cb85 None None] MFA Rules not processed for user `f2e6dffba870453fab836e0ccece7708`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:45.031828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cc2828d5-6d14-4d15-85b9-a3411903cb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:45.074027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:45.111128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 695/1395] 10.222.0.18 () {58 vars in 1063 bytes} [Wed Jul 20 09:28:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:45.118802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:45.120359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:45.134259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:45.225625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cbcb3-dfbb-4697-bf01-799e0ab93ffe None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:45.298564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:45.299124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:45.299984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:45.300258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:45.301889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:45.369022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:45.410735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:45.412218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca5c3b-572e-4451-abbb-6ac003d9856c service glance] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:45.430729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 701/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:46.140677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248ae5-76d4-4eda-95ed-06e2a0a5f23b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.141236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248ae5-76d4-4eda-95ed-06e2a0a5f23b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.141335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61248ae5-76d4-4eda-95ed-06e2a0a5f23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.208421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-61248ae5-76d4-4eda-95ed-06e2a0a5f23b None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:46.209637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-61248ae5-76d4-4eda-95ed-06e2a0a5f23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.300548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 696/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:46.347603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01cd3-5248-410a-be95-35ccd7604b8c None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.348103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01cd3-5248-410a-be95-35ccd7604b8c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.348323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff01cd3-5248-410a-be95-35ccd7604b8c None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.349835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 702/1398] 10.222.0.18 () {54 vars in 943 bytes} [Wed Jul 20 09:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:46.356191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.356548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.356895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.425397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:46.433268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] MFA Rules not processed for user `826539d8549b40ddacbe997a519691ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:46.434602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.448022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202cff6-979c-4195-a540-7e61a11978f5 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.448317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202cff6-979c-4195-a540-7e61a11978f5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.448600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202cff6-979c-4195-a540-7e61a11978f5 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.449986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 703/1399] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:28:46.455179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.455875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.455875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.522655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:46.524057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.550469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 697/1400] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Jul 20 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:46.561064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:46.571416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af29f-eba4-4e01-a12c-abb8b16ff0ba None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:46.632040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 704/1401] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:28:46.677930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:46.679059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.682483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:46.683143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.683454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.683733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.685402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.687962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:46.708829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:46.772227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:46.773829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f114294-a5bf-4967-9226-9a061a7cee1b service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:46.781868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 698/1402] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:28:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:28:46.797699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdb167-5361-4cab-b9de-afb4fcf71795 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:46.926179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:46.927059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:46.927445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:46.927790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:46.929582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:46.948982 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:47.034137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:47.035842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48920d49-e2fa-4480-82f1-cbc0d7a49119 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:47.056324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 705/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:49.225476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:49.225476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:49.225476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:49.228000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 699/1404] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:49.230775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd458f-d5cc-43e2-b00e-4fe05e4a7a10 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:49.232298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd458f-d5cc-43e2-b00e-4fe05e4a7a10 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:49.233065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd458f-d5cc-43e2-b00e-4fe05e4a7a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:49.312299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3dfd458f-d5cc-43e2-b00e-4fe05e4a7a10 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:49.313530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3dfd458f-d5cc-43e2-b00e-4fe05e4a7a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:49.415233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:49.422136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 706/1405] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:49.468008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:49.469536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:49.632486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:49.635568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3198-8ba0-4109-96e9-f71d706f6aef None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:49.775516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:49.776192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:49.776480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:49.776759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:49.778500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:49.893520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:49.895156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331f643-c3c2-4465-9752-d25c64482bec service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:49.915236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 700/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:52.003631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:52.004197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:52.004197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:52.005747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 707/1407] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:28:52.012667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c734-9abf-45d4-a7f2-7de383b2642d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:52.013120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c734-9abf-45d4-a7f2-7de383b2642d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:52.013314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c734-9abf-45d4-a7f2-7de383b2642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:52.016693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:52.099597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5312c734-9abf-45d4-a7f2-7de383b2642d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:28:52.101857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5312c734-9abf-45d4-a7f2-7de383b2642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:52.239377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 701/1408] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:28:52.297353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:52.299483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:52.470244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402a4a7-c20a-4af8-a71b-58b8a278ca92 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:52.547500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:52.604462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:52.605146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:52.605427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:52.605701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:52.607402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:52.713176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:52.714712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718814ea-df76-4903-b34c-8b88f8dd5665 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:52.739642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 708/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:02.236565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=64 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:02.733690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=64 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:09.708870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69334a5-6f82-45dc-8dca-1a45347f062f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:09.709415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69334a5-6f82-45dc-8dca-1a45347f062f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:09.709415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69334a5-6f82-45dc-8dca-1a45347f062f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:09.710794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 702/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Jul 20 09:29:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:09.716038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec153ea-9e5c-4966-ade1-328102b9201a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:09.716477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec153ea-9e5c-4966-ade1-328102b9201a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:09.716580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec153ea-9e5c-4966-ade1-328102b9201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:09.790287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fec153ea-9e5c-4966-ade1-328102b9201a None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:09.791518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fec153ea-9e5c-4966-ade1-328102b9201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:09.897884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 709/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 09:29:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:09.912173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665a38-3abf-411a-8d3d-a64b47258727 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:09.912468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665a38-3abf-411a-8d3d-a64b47258727 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:09.912745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea665a38-3abf-411a-8d3d-a64b47258727 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:09.914135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 703/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Jul 20 09:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:09.918834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b887d-bc14-40f6-98c6-bd7db764e9d5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:09.919121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b887d-bc14-40f6-98c6-bd7db764e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:09.919419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b887d-bc14-40f6-98c6-bd7db764e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:09.990055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:09.992569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0d9b887d-bc14-40f6-98c6-bd7db764e9d5 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:09.994344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0d9b887d-bc14-40f6-98c6-bd7db764e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:10.106802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 710/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 09:29:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:12.628429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:12.629018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:12.629018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:12.630359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:29:12.635831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab748-18a3-4285-8b9f-e860f3fa2837 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:12.636253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab748-18a3-4285-8b9f-e860f3fa2837 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:12.636390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ab748-18a3-4285-8b9f-e860f3fa2837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:12.699887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b36ab748-18a3-4285-8b9f-e860f3fa2837 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:12.701007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b36ab748-18a3-4285-8b9f-e860f3fa2837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:12.756505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:12.798216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:12.842630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:12.844214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:12.988567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fca6e-7e88-4f91-8e6e-b14ed4f2f0fe None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:13.063253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:13.104386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:13.105034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:13.105319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:13.105598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:13.107276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:13.194533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:13.196172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610dc72-cf34-4f33-85e3-af71bc9f914d service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:13.215071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 705/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:14.127358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:14.127894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:14.127894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:14.129209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:14.134147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e04fe-e0e4-44f6-b81e-90ebc140b80c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:14.134575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e04fe-e0e4-44f6-b81e-90ebc140b80c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:14.134721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e04fe-e0e4-44f6-b81e-90ebc140b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:14.208150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-098e04fe-e0e4-44f6-b81e-90ebc140b80c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:14.209348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-098e04fe-e0e4-44f6-b81e-90ebc140b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:14.235360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:14.304627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:29:14.456512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:14.458031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:14.602720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3063fa-b464-48a0-9394-e5cff12e4bcb None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:14.641177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:14.713823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:14.714386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:14.714670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:14.714944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:14.716604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:14.803080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:14.804715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36524b1-6199-47dc-b99a-86e6ce813c90 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:14.822665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 713/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:15.709638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:15.710193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:15.710193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:15.711546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:15.717615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947dcb-e8a2-4e6f-b599-cedc308a62fc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:15.718046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947dcb-e8a2-4e6f-b599-cedc308a62fc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:15.718160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4947dcb-e8a2-4e6f-b599-cedc308a62fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:15.770792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:15.792618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d4947dcb-e8a2-4e6f-b599-cedc308a62fc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:15.793818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d4947dcb-e8a2-4e6f-b599-cedc308a62fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:15.888884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:16.041059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:16.042575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:16.185708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec8f5a-e0f8-4a9d-ae2e-a16372bd5b5a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:16.198567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:16.300221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:16.300799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:16.301080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:16.301351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:16.302987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:16.390385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:16.392235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fdcf7-a76e-4955-b3b8-68feb0a0bcc1 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:16.410280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 708/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:17.295268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:17.295896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:17.295896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:17.297277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:17.302222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259b48-1330-49e4-9b8e-1c665b0451e7 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:17.302675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259b48-1330-49e4-9b8e-1c665b0451e7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:17.302778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7259b48-1330-49e4-9b8e-1c665b0451e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:17.321745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:17.377140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a7259b48-1330-49e4-9b8e-1c665b0451e7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:17.378346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a7259b48-1330-49e4-9b8e-1c665b0451e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:17.473645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:17.631551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:17.632998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:17.742045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:17.779775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80b1a6-d780-4a41-910e-47b703c7fa75 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:17.891928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:17.892476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:17.892753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:17.893028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:17.894677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:17.969473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:17.983141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:17.985377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c786c-634f-4700-b4e1-1665a8e97014 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:18.003370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 716/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:18.876755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:18.877427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:18.877427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:18.878668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:18.883894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c10db-e71e-40ea-8dbe-7a7c93f44a07 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:18.884350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c10db-e71e-40ea-8dbe-7a7c93f44a07 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:18.884465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c10db-e71e-40ea-8dbe-7a7c93f44a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:18.958344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5a5c10db-e71e-40ea-8dbe-7a7c93f44a07 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:18.959588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5a5c10db-e71e-40ea-8dbe-7a7c93f44a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:19.054853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:19.205691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:19.207133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:19.292157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:19.353439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27beb22-ce31-4c3b-bc56-c6378b671b84 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:19.466042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:19.466633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:19.466911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:19.467184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:19.468872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:19.516505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:19.557318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:19.558912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe827e-e6cb-482e-beec-1c850676fea8 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:19.579417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 711/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:20.389319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:20.389948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:20.390081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:20.391445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:20.396261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbb40-ad0c-42e9-969b-3593f9e8eded None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:20.396661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbb40-ad0c-42e9-969b-3593f9e8eded None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:20.396823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbb40-ad0c-42e9-969b-3593f9e8eded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:20.472230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-872fbb40-ad0c-42e9-969b-3593f9e8eded None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:20.473434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-872fbb40-ad0c-42e9-969b-3593f9e8eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:20.566992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:20.719507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:20.720944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:20.770150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:20.867062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba30ad-2257-43c0-918b-41640ae98dbf None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:20.977301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:20.977882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:20.978163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:20.978435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:20.980231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:20.987077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:21.069544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:21.071164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498d80c-7d85-47fc-8ac7-ad60940d2767 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:21.089362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 719/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:21.896577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:21.897136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:21.897136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:21.898511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:21.903399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b90-7f54-4c5c-822f-d0dae253b291 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:21.903829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b90-7f54-4c5c-822f-d0dae253b291 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:21.903969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e9b90-7f54-4c5c-822f-d0dae253b291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:21.978798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fe2e9b90-7f54-4c5c-822f-d0dae253b291 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:21.980014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fe2e9b90-7f54-4c5c-822f-d0dae253b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:22.046628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:22.075418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:22.235379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:22.236904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:22.254228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:22.382029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725951-359c-4efd-98d6-44b1d773aec5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:22.469260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:22.495527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:22.496170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:22.496447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:22.496719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:22.498872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:22.585814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:22.587434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f0d44-ee2b-4d78-a5c1-5427a5aa505d service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:22.605051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 714/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:23.409741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:23.410346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:23.410346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:23.411686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:23.416507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08f52c-d2a9-471d-8d9b-ab9e598add70 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:23.416910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08f52c-d2a9-471d-8d9b-ab9e598add70 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:23.417063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08f52c-d2a9-471d-8d9b-ab9e598add70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:23.492532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ce08f52c-d2a9-471d-8d9b-ab9e598add70 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:23.493741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ce08f52c-d2a9-471d-8d9b-ab9e598add70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:23.534306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:23.590059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:23.740827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:23.742259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:23.887762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358c8e6-9d58-47bc-a482-08746eee1db7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:23.940533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:23.999774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:24.000345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:24.000626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:24.000900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:24.002530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:24.090298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:24.092591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc98252-323d-490c-bf37-14579b6e0a93 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:24.110970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 722/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:24.925814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:24.926394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:24.926394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:24.927778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:24.932558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c14db6-eddd-49e1-b2a1-fab0f3d9e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:24.932991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c14db6-eddd-49e1-b2a1-fab0f3d9e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:24.933102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c14db6-eddd-49e1-b2a1-fab0f3d9e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:25.008201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e9c14db6-eddd-49e1-b2a1-fab0f3d9e0d3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:25.009386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e9c14db6-eddd-49e1-b2a1-fab0f3d9e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:25.011391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:25.105023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:25.256546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:25.258064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:25.402063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1da04-4c84-4b72-9f81-fddcc94da8eb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:25.430209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:25.517087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408356c2-070e-4c0e-942e-8428627707db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:25.517735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408356c2-070e-4c0e-942e-8428627707db service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:25.518018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408356c2-070e-4c0e-942e-8428627707db service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:25.518294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408356c2-070e-4c0e-942e-8428627707db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:25.519981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-408356c2-070e-4c0e-942e-8428627707db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:25.609939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408356c2-070e-4c0e-942e-8428627707db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:25.611574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408356c2-070e-4c0e-942e-8428627707db service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:25.630267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 717/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:27.582152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:27.582716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:27.582716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:27.584116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:27.589358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a443-cdc1-4901-80d5-97ac3fd20629 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:27.590269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a443-cdc1-4901-80d5-97ac3fd20629 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:27.590608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a443-cdc1-4901-80d5-97ac3fd20629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:27.631045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:27.663261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d4f7a443-cdc1-4901-80d5-97ac3fd20629 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:27.664489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d4f7a443-cdc1-4901-80d5-97ac3fd20629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:27.763375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:27.803183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:27.804745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:27.907823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc30de-2b6a-42a6-976b-56ffd45e92a8 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:27.940848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:27.983057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:27.983664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:27.983950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:27.984226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:27.985876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:28.095173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:28.096663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046763a-dec1-48d7-ad6c-adc8d40055b3 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:28.117133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 725/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:30.299884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:30.300403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:30.300523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:30.301821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:30.306995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990c72-792f-4eab-baa1-dab6016281f4 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:30.307503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990c72-792f-4eab-baa1-dab6016281f4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:30.308213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7990c72-792f-4eab-baa1-dab6016281f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:30.373098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a7990c72-792f-4eab-baa1-dab6016281f4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:30.374284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a7990c72-792f-4eab-baa1-dab6016281f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:30.386654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:30.470853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:30.508631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:30.510159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:30.613499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222cdfe-aac5-445f-a91a-9a31b8cb5ac4 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:30.626347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:30.691139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:30.691812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:30.692117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:30.692388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:30.694010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:30.812244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:30.813678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de726608-3a95-49ed-9cd9-5e52a4cad055 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:30.832514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 720/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:33.261591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:33.262233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:33.262233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:33.263555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:33.268799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689b65-d131-4d17-89c8-48d4c59cd3a2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:33.269220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689b65-d131-4d17-89c8-48d4c59cd3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:33.269359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7689b65-d131-4d17-89c8-48d4c59cd3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:33.315806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:33.334313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c7689b65-d131-4d17-89c8-48d4c59cd3a2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:33.335551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c7689b65-d131-4d17-89c8-48d4c59cd3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:33.429828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:33.471198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:33.472758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:33.576519 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186243e0-caaa-49c8-85a9-f9461ecc58df None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:33.616045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:33.651531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:33.652179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:33.652462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:33.652737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:33.654391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:33.764754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:33.766192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2a25f-671e-4f35-a6f5-55409b5f2271 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:33.784046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 728/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:36.013310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:36.013886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:36.013886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:36.015271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:36.020489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72fefa-4f9e-43d6-86c6-8269f2355f7c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:36.020922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72fefa-4f9e-43d6-86c6-8269f2355f7c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:36.021035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72fefa-4f9e-43d6-86c6-8269f2355f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:36.084193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3b72fefa-4f9e-43d6-86c6-8269f2355f7c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:36.085324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3b72fefa-4f9e-43d6-86c6-8269f2355f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:36.106188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:36.183362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:36.223122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:36.224712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:36.326574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a68dd6-a1a1-455b-8606-47e4b3dad85b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:36.346506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:36.398232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:36.399510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:36.399984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:36.400258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:36.401881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:36.508900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:36.510464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1b271-d308-477e-bc1a-13614d5934b0 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:36.528570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 723/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:38.706578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:38.707131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:38.707131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:38.708449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:38.714123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341df9-61fb-42bc-b2c3-9ba8d2878f07 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:38.714543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341df9-61fb-42bc-b2c3-9ba8d2878f07 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:38.714688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1341df9-61fb-42bc-b2c3-9ba8d2878f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:38.777122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:38.780097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f1341df9-61fb-42bc-b2c3-9ba8d2878f07 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:38.781926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f1341df9-61fb-42bc-b2c3-9ba8d2878f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:38.877425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:38.917229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:38.918822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:39.019968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f43f8-e196-4307-89b2-16e426067a30 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:39.068653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:39.093149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:39.093786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:39.094069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:39.094345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:39.096003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:39.203053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:39.204522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbfd64-e854-4696-a9b6-02b4b49476ef service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:39.222735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 731/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:41.365769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:41.366537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:41.366537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:41.367833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:41.373200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b071f-de7d-4b92-a9d3-0ba44917eef4 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:41.373649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b071f-de7d-4b92-a9d3-0ba44917eef4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:41.373750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b071f-de7d-4b92-a9d3-0ba44917eef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:41.440219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e16b071f-de7d-4b92-a9d3-0ba44917eef4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:41.441450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e16b071f-de7d-4b92-a9d3-0ba44917eef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:41.471343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:41.539109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:41.578176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:41.579766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:41.682988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f301c-3776-4aea-943e-d7b848426dc9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:41.712329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:41.757056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:41.757880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:41.758159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:41.758432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:41.760105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:41.870557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:41.872013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382efbb-105b-4ef7-9e8d-50e302906c84 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:41.889892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 726/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:43.984625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:43.985134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:43.985249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:43.986573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:43.992035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06acca3-2165-49e3-9ad4-3da4a8b23635 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:43.992479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06acca3-2165-49e3-9ad4-3da4a8b23635 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:43.992579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06acca3-2165-49e3-9ad4-3da4a8b23635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.057649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e06acca3-2165-49e3-9ad4-3da4a8b23635 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:44.058766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e06acca3-2165-49e3-9ad4-3da4a8b23635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.067222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:44.159016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:44.198069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:44.199625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.304016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9acfea-d402-4897-bae7-952988b4c291 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:44.357797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:44.376136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:44.376776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:44.377057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:44.377327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.378960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.487276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:44.488737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4afd-9ed1-4382-a501-a15047218f98 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:44.506969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 734/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:46.678263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:46.678835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:46.678835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:46.680211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:46.685920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780c887-54c3-4804-8f67-818766b0aff0 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:46.686357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780c887-54c3-4804-8f67-818766b0aff0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:46.686475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780c887-54c3-4804-8f67-818766b0aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:46.754108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5780c887-54c3-4804-8f67-818766b0aff0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:46.755351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5780c887-54c3-4804-8f67-818766b0aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:46.794174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:46.852350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:46.890605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:46.892177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:46.994023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b023dd6-0d79-43a6-995f-0768efa159af None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:47.030203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:47.066029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:47.066737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:47.067046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:47.067356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:47.069582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:47.182470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:47.183921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbb9404-6942-4ccf-92d3-d1bc61f37d2b service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:47.201761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 729/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:49.265787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:49.266335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:49.266335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:49.267743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:49.273980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac3eb-b326-42b7-a6eb-38b8bbc43292 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:49.274394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac3eb-b326-42b7-a6eb-38b8bbc43292 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:49.274539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcac3eb-b326-42b7-a6eb-38b8bbc43292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:49.341971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7bcac3eb-b326-42b7-a6eb-38b8bbc43292 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:49.343187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7bcac3eb-b326-42b7-a6eb-38b8bbc43292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:49.360922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:49.438498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:49.475642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:49.477165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:49.579478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7888e-5e25-4cfc-aa1e-2e972a55be3d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:49.646267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:49.653012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:49.653648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:49.653923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:49.654192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:49.655835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:49.763850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:49.765336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeff9d-703a-409d-a7ae-0ee279f70799 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:49.783106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 737/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:51.830273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:51.830798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:51.830798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:51.832769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:29:51.838274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4af-f882-44c5-a3c9-a55c5e858806 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:51.838694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4af-f882-44c5-a3c9-a55c5e858806 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:51.838837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4af-f882-44c5-a3c9-a55c5e858806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:51.902821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b6ade4af-f882-44c5-a3c9-a55c5e858806 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:29:51.904059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b6ade4af-f882-44c5-a3c9-a55c5e858806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:51.951993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:52.000599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:29:52.038492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:52.040084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:52.151203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1158ed-f20c-4ef0-ba97-aa92d0e4dbe0 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:52.198895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:52.225847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:52.226497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:52.226783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:52.227060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:52.228944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:52.336951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:52.338390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed0b52-181f-4335-8df0-6773e79363b3 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:52.356064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 732/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:30:01.998725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=22 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:02.353771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=66 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:12.206454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcad88-e860-4ac6-9264-0d08ffc4405e None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:12.207079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcad88-e860-4ac6-9264-0d08ffc4405e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:12.207079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcad88-e860-4ac6-9264-0d08ffc4405e None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:12.208342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 739/1471] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:12.213703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fec1-fb52-4606-9c80-f6dad437f3be None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:12.214149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fec1-fb52-4606-9c80-f6dad437f3be None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:12.214261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fec1-fb52-4606-9c80-f6dad437f3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:12.279049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-efd0fec1-fb52-4606-9c80-f6dad437f3be None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:12.280340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-efd0fec1-fb52-4606-9c80-f6dad437f3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:12.375300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 733/1472] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:12.458762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21402177-4f62-4b35-8f46-a01160016441 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:12.458993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21402177-4f62-4b35-8f46-a01160016441 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:12.459320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21402177-4f62-4b35-8f46-a01160016441 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:12.460690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 740/1473] 10.222.0.18 () {54 vars in 966 bytes} [Wed Jul 20 09:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:12.465404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad6bb-a64c-406d-b58b-27b1581a0a85 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:12.465696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad6bb-a64c-406d-b58b-27b1581a0a85 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:12.465978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad6bb-a64c-406d-b58b-27b1581a0a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:12.540881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-be5ad6bb-a64c-406d-b58b-27b1581a0a85 None None] MFA Rules not processed for user `f2e6dffba870453fab836e0ccece7708`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:12.542089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-be5ad6bb-a64c-406d-b58b-27b1581a0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:12.563263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:12.620304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 734/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Wed Jul 20 09:30:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:12.628551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21402177-4f62-4b35-8f46-a01160016441 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:12.630090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-21402177-4f62-4b35-8f46-a01160016441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:12.746707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21402177-4f62-4b35-8f46-a01160016441 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:12.819574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:12.820219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:12.820505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:12.820778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:12.822428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:12.875639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:12.930878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:12.932389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edc6cb-4980-480e-891f-f7e80c4d9e3a service glance] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:12.951730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 741/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:15.112537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:15.113104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:15.113104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:15.114480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 735/1476] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:15.119890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2985f1-6a88-4689-915e-95ba40168c6c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:15.120343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2985f1-6a88-4689-915e-95ba40168c6c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:15.120445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2985f1-6a88-4689-915e-95ba40168c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:15.185894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fb2985f1-6a88-4689-915e-95ba40168c6c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:15.187105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fb2985f1-6a88-4689-915e-95ba40168c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:15.284104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 742/1477] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:30:15.307909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:15.309960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:15.435538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961aa1db-abee-452c-9584-3fce69fe7080 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:15.483175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:15.528486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:15.529152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:15.529437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:15.529717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:15.530440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:15.532241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:30:15.532241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:30:15.532241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:30:15.533399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07aa83-354f-4f9f-9401-80e23cbbfb5a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:15.552841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 736/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Jul 20 09:30:15 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:30:17.613633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:17.614143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:17.614310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:17.615739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 743/1479] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:17.621043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be437d-565f-498e-8e0a-d63ad8926e67 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:17.621495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be437d-565f-498e-8e0a-d63ad8926e67 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:17.621627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be437d-565f-498e-8e0a-d63ad8926e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:17.688482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-48be437d-565f-498e-8e0a-d63ad8926e67 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:17.689692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-48be437d-565f-498e-8e0a-d63ad8926e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:17.790360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 737/1480] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:17.835042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:17.836652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:17.848439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:17.988833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe6a6f-f467-4a8c-a99e-41afefc6f132 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:18.073299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:18.107218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:18.107891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:18.108166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:18.108434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:18.110064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:18.202438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:18.204099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b18aee-1c03-4a2d-b4e0-79f5a18b051d service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:18.222227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 744/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:20.431370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:20.432049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:20.432049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:20.433972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 738/1482] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:20.439316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4570b-56ac-4323-b758-ab7d85134deb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:20.439778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4570b-56ac-4323-b758-ab7d85134deb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:20.439894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4570b-56ac-4323-b758-ab7d85134deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:20.503333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7fe4570b-56ac-4323-b758-ab7d85134deb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:20.504555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7fe4570b-56ac-4323-b758-ab7d85134deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:20.550518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:20.600405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 745/1483] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:20.637068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:20.638626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:20.663079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:20.743722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596fe8d-e441-4632-bec3-9fb8bfdcbb8f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:20.815873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:20.816520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:20.816806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:20.817079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:20.818711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:20.894114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:20.929436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:20.930869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d74e7-4dce-4813-a1cd-2707ec722c76 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:20.949225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 739/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:23.050589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:23.051138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:23.051252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:23.053254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 746/1485] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:23.058866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befda0-4dfa-44a6-b1c6-f161df4bba47 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:23.059423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befda0-4dfa-44a6-b1c6-f161df4bba47 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:23.059540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23befda0-4dfa-44a6-b1c6-f161df4bba47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:23.125242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-23befda0-4dfa-44a6-b1c6-f161df4bba47 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:23.126456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-23befda0-4dfa-44a6-b1c6-f161df4bba47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:23.223184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 740/1486] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:23.263145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:23.264724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:23.367454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6298a6-b03b-4cfd-933e-81248cc5b737 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:23.442198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:23.444301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:23.445001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:23.445323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:23.445604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:23.447795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:23.556364 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:23.558145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd23f6-e084-449a-ab0f-ec93963f8976 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:23.575800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 747/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:25.745876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:25.746442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:25.746442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:25.747961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 741/1488] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:25.753194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d98d-40cc-4292-98b1-ca29c3e1fc1e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:25.753613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d98d-40cc-4292-98b1-ca29c3e1fc1e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:25.753769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547d98d-40cc-4292-98b1-ca29c3e1fc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:25.818133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9547d98d-40cc-4292-98b1-ca29c3e1fc1e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:25.819390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9547d98d-40cc-4292-98b1-ca29c3e1fc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:25.871861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:25.915857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 748/1489] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:25.957946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:25.959542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:25.995620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:26.063887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579caaf-09ec-440d-a70d-77bbba79b457 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:26.143719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:26.144839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:26.145352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:26.145846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:26.149139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:26.235673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:26.266419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:26.267888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd23b7f-b6c9-47e3-b95a-7d34a21bea02 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:26.285740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 742/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:28.522370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:28.522917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:28.522917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:28.524330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 749/1491] 10.222.0.18 () {54 vars in 926 bytes} [Wed Jul 20 09:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:30:28.530231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1f75-dfd7-4a93-a182-4ff2434b935d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:28.530682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1f75-dfd7-4a93-a182-4ff2434b935d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:28.530783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1f75-dfd7-4a93-a182-4ff2434b935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:28.596343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c48a1f75-dfd7-4a93-a182-4ff2434b935d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:30:28.597553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c48a1f75-dfd7-4a93-a182-4ff2434b935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:28.692214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 743/1492] 10.222.0.18 () {58 vars in 1023 bytes} [Wed Jul 20 09:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:30:28.732016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:30:28.733541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:28.837107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411992e3-86da-4724-ba50-ce8ecedc275d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:30:28.909679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:30:28.910322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:30:28.910605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:30:28.910880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:30:28.912557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:30:28.921643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:29.022542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:30:29.024057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67108b27-2ae4-498c-b54f-effcf2983372 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:30:29.042205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 750/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:30:38.690316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=82 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:30:39.039375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=50 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:57.027061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:57.027692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:57.027692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:57.100431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:31:57.101616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:57.199152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 744/1494] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:31:57.207425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:57.207920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:57.208048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:57.280631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:31:57.281738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:57.377083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 751/1495] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:31:57.381882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:57.383133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:57.395107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae6a53-3cd1-4d47-ba03-1a3bb4258729 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:57.573579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:57.608268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:57.608920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:57.609198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:57.609467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:57.610168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:31:57.610479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:57.611537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:57.611537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:31:57.611537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:31:57.612400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:57.623908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 745/1496] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:31:57.632050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:57.642088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d7979-32f3-45a2-be28-b4b1f5e5871b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:57.705104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:57.741437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:57.742108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:57.742392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:57.742698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:57.743437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:57.745215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:57.745215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:31:57.745215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:31:57.746436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:57.800753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 752/1497] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:31:57 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:31:57.804981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:57.806574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:57.840123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83914e46-3dc1-41a8-bed8-3c8fe8b43afe admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:57.933590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:57.934312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:57.934591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:57.934860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:57.935577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:57.936585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:57.936585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:31:57.936585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:31:57.937771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:57.977661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 746/1498] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:31:57 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:31:57.981929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:57.983480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.044873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03b071-b537-40d4-b135-6aa09c54c836 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:58.130554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:58.143817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:58.144513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.144794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.145069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.145791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:58.146801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:58.146801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:31:58.146801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:31:58.147736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:58.160708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 753/1499] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:31:58.165021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:58.166532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.178053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d21cca-a129-48b9-8443-95d3a11206b1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:58.203265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:58.271857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:58.272520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.272797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.273073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.273783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:58.274782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:58.274782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:31:58.274782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:31:58.276293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edbbfd-e85c-4830-bcf8-855501603f9a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:58.288199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 747/1500] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:31:58.292887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:58.294410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.328376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef2f9b-f06a-4fa4-9ff9-2c8914c29d12 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:58.420808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:58.421611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.421887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.422159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] PATH_INFO: `/v3/projects/cd05287157044b35b4895c26a79d5c80/users/1a3fa3da68f942e98ecb590531f1d104/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.445196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd05287157044b35b4895c26a79d5c80, user_id=1a3fa3da68f942e98ecb590531f1d104, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:58.446250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:58.446250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:31:58.446250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:31:58.449119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814ec8-412f-4534-b903-845f97478e4a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:58.474116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 754/1501] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:31:58 2022] PUT /identity/v3/projects/cd05287157044b35b4895c26a79d5c80/users/1a3fa3da68f942e98ecb590531f1d104/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:31:58.486052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac8dd-705e-4856-96d5-1fe6eec4cfba None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.486340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac8dd-705e-4856-96d5-1fe6eec4cfba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.486616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebac8dd-705e-4856-96d5-1fe6eec4cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.542911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7ebac8dd-705e-4856-96d5-1fe6eec4cfba None None] MFA Rules not processed for user `1a3fa3da68f942e98ecb590531f1d104`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:31:58.544142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7ebac8dd-705e-4856-96d5-1fe6eec4cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.622116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 748/1502] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:31:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:31:58.635872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.636157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.636429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.637814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 755/1503] 10.222.0.18 () {54 vars in 964 bytes} [Wed Jul 20 09:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:31:58.642654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96852-c8cc-4da0-9396-1807499b13b0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.642946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96852-c8cc-4da0-9396-1807499b13b0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.643226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96852-c8cc-4da0-9396-1807499b13b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.673396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:58.717886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-12b96852-c8cc-4da0-9396-1807499b13b0 None None] MFA Rules not processed for user `267cd2f0b4d442ee9ab2ee668394472c`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:31:58.719117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-12b96852-c8cc-4da0-9396-1807499b13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.793692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 749/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Jul 20 09:31:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:31:58.801426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:58.802932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:58.851738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:58.905219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc51646-7261-4b11-b707-a9a5902263c1 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:58.976924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:58.977499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:58.977787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:58.978055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:58.980272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:59.076776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:59.078209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:59.081665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:59.097341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 756/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:31:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:31:59.139707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87d5a3-45b9-4994-a133-b19ced573ae4 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:59.139989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87d5a3-45b9-4994-a133-b19ced573ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:59.140262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87d5a3-45b9-4994-a133-b19ced573ae4 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:59.141569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 750/1506] 10.222.0.18 () {52 vars in 829 bytes} [Wed Jul 20 09:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:31:59.145150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:59.146372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:59.364833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f206-4507-4b55-a9fc-9e4aff0b6b64 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:59.457359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:59.458232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:59.458510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:59.458780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] PATH_INFO: `/v3/users/1a3fa3da68f942e98ecb590531f1d104` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:59.471916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3fa3da68f942e98ecb590531f1d104)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:59.472950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:31:59.472950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:31:59.472950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:31:59.474160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:59.484117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:59.534753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3fa3da68f942e98ecb590531f1d104'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:31:59.544882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3fa3da68f942e98ecb590531f1d104'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:31:59.563065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3fa3da68f942e98ecb590531f1d104'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:31:59.563541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3fa3da68f942e98ecb590531f1d104'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:31:59.575110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3fa3da68f942e98ecb590531f1d104'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:31:59.580696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 757/1507] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:31:59 2022] DELETE /identity/v3/users/1a3fa3da68f942e98ecb590531f1d104 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:31:59.585137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:59.585609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:59.585713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:59.640943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:31:59.642056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:59.715206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:59.728661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 751/1508] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:31:59.738085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:59.739370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:59.838826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ddec2-f343-4dcf-accb-c6914d9e1a21 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:59.894720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:59.914671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:59.915342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:59.915630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:59.916466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:59.918092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:32:00.025932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:32:00.027387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7a2a-0098-496e-833d-7a151fb42bc6 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:32:00.048868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 758/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:32:00.428897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:32:00.430166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:32:00.445903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c37644-ff6a-4736-9356-e1d40e9493fa None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:32:00.544968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:32:00.545863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:32:00.546144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:32:00.546427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] PATH_INFO: `/v3/projects/cd05287157044b35b4895c26a79d5c80` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:32:00.556505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd05287157044b35b4895c26a79d5c80)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:32:00.557399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:32:00.557399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:32:00.557399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:32:00.558581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:32:00.615955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd05287157044b35b4895c26a79d5c80', 'request_id': 'req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:32:00.622000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd05287157044b35b4895c26a79d5c80', 'request_id': 'req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:32:00.631826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd05287157044b35b4895c26a79d5c80', 'request_id': 'req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:32:00.661134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] The token cache is being invalidate because project cd05287157044b35b4895c26a79d5c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:32:00.661419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:32:00.661847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-cbc66ad7-1cb2-450b-a8a9-2b3cbf7d9419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:32:00.663282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 752/1510] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:32:00 2022] DELETE /identity/v3/projects/cd05287157044b35b4895c26a79d5c80 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:32:10.046540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=62 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:32:10.659064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:33:41.165893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:41.166533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:41.166533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:41.167903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:41.172690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9dc83-5487-4e9a-942f-4d723d660ae9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:41.173148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9dc83-5487-4e9a-942f-4d723d660ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:41.173255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9dc83-5487-4e9a-942f-4d723d660ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:41.251878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-afc9dc83-5487-4e9a-942f-4d723d660ae9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:41.253099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-afc9dc83-5487-4e9a-942f-4d723d660ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:41.356042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:41.514280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:41.516456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:41.635498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ad94-1f3e-4ffe-a391-ad6543f00231 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:41.640965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:41.641609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:41.641892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:41.642164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:41.650869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:41.652477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57479b04-f069-49fa-a25b-d3ce17548df0 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:41.662014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 760/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:33:42.498620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c8be-3fa6-4d68-bfd0-b6dead4e091b None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:42.499106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c8be-3fa6-4d68-bfd0-b6dead4e091b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:42.499203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0c8be-3fa6-4d68-bfd0-b6dead4e091b None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:42.500645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:42.505946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef476ee-d853-4a84-9ced-dbcc7b691569 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:42.506402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef476ee-d853-4a84-9ced-dbcc7b691569 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:42.506507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef476ee-d853-4a84-9ced-dbcc7b691569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:42.524596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6ef476ee-d853-4a84-9ced-dbcc7b691569 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:42.525783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6ef476ee-d853-4a84-9ced-dbcc7b691569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:42.533556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:42.683416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0c8be-3fa6-4d68-bfd0-b6dead4e091b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:42.699151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0c8be-3fa6-4d68-bfd0-b6dead4e091b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:42.704925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:42.705564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:42.705907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:42.706180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:42.714736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:42.716358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534adb5c-1bf9-4a3c-ab73-a88dd1582654 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:42.727268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 755/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:33:42.775770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d081-44c6-4357-95f8-28bda3725043 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:42.776082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d081-44c6-4357-95f8-28bda3725043 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:42.776336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432d081-44c6-4357-95f8-28bda3725043 None None] PATH_INFO: `/v3` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:42.777676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 762/1517] 10.222.0.17 () {54 vars in 925 bytes} [Wed Jul 20 09:33:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:33:42.791611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6022-f172-406d-b5c0-7f0f8e4f399a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:42.791904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6022-f172-406d-b5c0-7f0f8e4f399a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:42.792183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6022-f172-406d-b5c0-7f0f8e4f399a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:42.853307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f39c6022-f172-406d-b5c0-7f0f8e4f399a None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:42.854511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f39c6022-f172-406d-b5c0-7f0f8e4f399a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:42.872339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:33:42.925894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 756/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Jul 20 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:46.721299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138a30-78e7-45e2-80f0-47e9e09c6b7b None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:46.721807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138a30-78e7-45e2-80f0-47e9e09c6b7b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:46.721807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03138a30-78e7-45e2-80f0-47e9e09c6b7b None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:46.723135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 763/1519] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jul 20 09:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:46.728888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ac01-77f8-47c5-8569-167bb281e64b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:46.729301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ac01-77f8-47c5-8569-167bb281e64b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:46.729444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1ac01-77f8-47c5-8569-167bb281e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:46.751250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-88a1ac01-77f8-47c5-8569-167bb281e64b None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:46.752477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-88a1ac01-77f8-47c5-8569-167bb281e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:46.761193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 757/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jul 20 09:33:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:46.768617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03138a30-78e7-45e2-80f0-47e9e09c6b7b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:46.781909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03138a30-78e7-45e2-80f0-47e9e09c6b7b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:46.787654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:46.788279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:46.788560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:46.788836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:46.796945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:46.798512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ded93b-7396-415b-807a-1f2f90c088ab service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:46.808582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 764/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:33:53.942759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d05e-eb34-43f6-8e9d-af1413a0a3f6 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:53.943186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d05e-eb34-43f6-8e9d-af1413a0a3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:53.943504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82d05e-eb34-43f6-8e9d-af1413a0a3f6 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:53.944895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:53.949659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eee4-33f3-45c6-bfad-3a1a5a6934b9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:53.950102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eee4-33f3-45c6-bfad-3a1a5a6934b9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:53.950216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eee4-33f3-45c6-bfad-3a1a5a6934b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:53.968385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ce49eee4-33f3-45c6-bfad-3a1a5a6934b9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:53.969573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ce49eee4-33f3-45c6-bfad-3a1a5a6934b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:53.977602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:33:54.133617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82d05e-eb34-43f6-8e9d-af1413a0a3f6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:54.147806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82d05e-eb34-43f6-8e9d-af1413a0a3f6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:54.153141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:54.153771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:54.154047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:54.154318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:54.163106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:54.164702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33f647-1f97-4b7c-80bd-83fcec57f62c service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:54.173890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 759/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:33:54.999625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e982-fc25-4528-ab88-7162b409d7cc None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:55.000197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e982-fc25-4528-ab88-7162b409d7cc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:55.000197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e982-fc25-4528-ab88-7162b409d7cc None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:55.001608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:55.006608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f85a-379a-43fd-95cf-89f24534e041 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:55.007057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f85a-379a-43fd-95cf-89f24534e041 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:55.007159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90f85a-379a-43fd-95cf-89f24534e041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:55.026577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ae90f85a-379a-43fd-95cf-89f24534e041 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:55.027814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ae90f85a-379a-43fd-95cf-89f24534e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:55.036567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:55.189382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e982-fc25-4528-ab88-7162b409d7cc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:55.204367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e982-fc25-4528-ab88-7162b409d7cc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:55.210567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:55.211202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:55.211512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:55.211813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:55.220003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:55.221573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63949da7-f2fb-497c-a24d-aea22dcffd56 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:55.232037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 767/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:33:59.209156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:59.209779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:59.209779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:59.211065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:33:59.216227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06e1bc-b270-4d4e-947f-e3e7428b7aa2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:59.216634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06e1bc-b270-4d4e-947f-e3e7428b7aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:59.216797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06e1bc-b270-4d4e-947f-e3e7428b7aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:59.234936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4c06e1bc-b270-4d4e-947f-e3e7428b7aa2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:33:59.236165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4c06e1bc-b270-4d4e-947f-e3e7428b7aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:59.244722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:33:59.317931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:33:59.319722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:33:59.396405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf407a5-9834-4e2d-b357-38b7c63ee56a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:33:59.399940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:33:59.400579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:33:59.400865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:33:59.401144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:33:59.410187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:33:59.412107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c95ed2-b2f2-4a72-9002-af172da3fb79 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:33:59.421393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 762/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:34:01.455723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:01.456185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:01.456305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:01.458319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:34:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:34:01.463644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede540a8-d301-47a6-8e74-4850c9e5d0f9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:01.464104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede540a8-d301-47a6-8e74-4850c9e5d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:01.464204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede540a8-d301-47a6-8e74-4850c9e5d0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:01.482631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ede540a8-d301-47a6-8e74-4850c9e5d0f9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:34:01.483856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ede540a8-d301-47a6-8e74-4850c9e5d0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:01.492233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:34:01.531193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:34:01.532958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:01.547520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acc71e-4301-4984-b11b-e40359e26b8f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:34:01.551277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:34:01.551964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:01.552249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:01.552527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:01.562438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:34:01.563910 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733df9e-c531-451c-ace3-93fe4b522957 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:34:01.574196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 770/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:34:04.973756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:04.974349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:04.974349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:04.975722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:34:04.980840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cf05e-d54a-4fa8-9f58-732ace2c20a1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:04.981290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cf05e-d54a-4fa8-9f58-732ace2c20a1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:04.981429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cf05e-d54a-4fa8-9f58-732ace2c20a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:04.999726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6a9cf05e-d54a-4fa8-9f58-732ace2c20a1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:34:05.000909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6a9cf05e-d54a-4fa8-9f58-732ace2c20a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:05.009583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:34:05.332271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:34:05.334064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:05.396753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:34:05.410841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930386e-5ea6-403c-8109-421e7e929f83 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:34:05.414413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:34:05.415062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:05.415369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:05.415648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:05.424424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:34:05.425903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77262-a675-4932-96e2-5cf20a77408b service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:34:05.435701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 765/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:34:07.514770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918236-3672-4323-856a-28c420b23797 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:07.515346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918236-3672-4323-856a-28c420b23797 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:07.515346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918236-3672-4323-856a-28c420b23797 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:07.516752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:34:07.522506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f2b6-10e8-4e79-93d2-c6e4584633b5 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:07.522958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f2b6-10e8-4e79-93d2-c6e4584633b5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:07.523070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f2b6-10e8-4e79-93d2-c6e4584633b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:07.541768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e669f2b6-10e8-4e79-93d2-c6e4584633b5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:34:07.542960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e669f2b6-10e8-4e79-93d2-c6e4584633b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:07.550787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:34:07.857034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918236-3672-4323-856a-28c420b23797 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:34:07.858739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f5918236-3672-4323-856a-28c420b23797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:34:07.872431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918236-3672-4323-856a-28c420b23797 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:34:07.875888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:34:07.876517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:34:07.876796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:34:07.877069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:34:07.887005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:34:07.888463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba06d69-f563-4c82-9365-c73f4b106374 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:34:07.898619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 773/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:34:17.536969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=11 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:34:17.893554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:37.746463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.747110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.747110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:37.781612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:37.782819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:37.793961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 767/1540] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:37.803336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0bf4-b3c0-4bea-a0a3-3bb215923a51 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.804393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0bf4-b3c0-4bea-a0a3-3bb215923a51 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.805153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0bf4-b3c0-4bea-a0a3-3bb215923a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:37.833416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-318e0bf4-b3c0-4bea-a0a3-3bb215923a51 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:37.835776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-318e0bf4-b3c0-4bea-a0a3-3bb215923a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:37.846733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 774/1541] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:37.851998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:37.865676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37906c58-aa8c-438c-b410-b75e9294359a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:37.878305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:37.879009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.879308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.879594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:37.882294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:37.882620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:37.888542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:37.905723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 768/1542] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:37.912822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc85ea-6208-478c-bfdf-0c2cd5e05430 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.913488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc85ea-6208-478c-bfdf-0c2cd5e05430 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.914001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc85ea-6208-478c-bfdf-0c2cd5e05430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:37.915348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:37.927343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be063230-72cb-43e4-a495-afc02f0ca4ed admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:37.942721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:37.944033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.944316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.944599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:37.945344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:37.946606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:37.946606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:35:37.946606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:35:37.947908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:37.956388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-46cc85ea-6208-478c-bfdf-0c2cd5e05430 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:37.957941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-46cc85ea-6208-478c-bfdf-0c2cd5e05430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:37.977593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 775/1543] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:37.986722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:37.987402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:37.987900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.013116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 769/1544] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:37 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:35:38.027315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.029940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.031903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.116520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164e0a4-bef1-4b2b-afd3-80788db43cdf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.121274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 776/1545] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:38.126232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.137502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.137502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.137502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.137502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.137958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.138523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.155882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6808-9db2-4e48-9a86-2098b5686e0f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.168052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.169148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.169759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.170274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.171329 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.171870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.173108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:38.173108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:35:38.173108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:35:38.174485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a4e65-dad2-45de-8ba8-7f8957fe50d4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.183372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 770/1546] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:35:38.184433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.203839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166235ed-3db0-4eb0-a8eb-d75e998ef358 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.206537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 777/1547] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:38.208091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.208738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.209254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.224012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.226190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.226190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.226190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.226642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.228370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b0f51-4014-4de3-94e6-ee5b20bd6e88 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.244827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.246590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.251318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 771/1548] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:38.251318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.251318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.251318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.264063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 778/1549] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.265377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.283170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510547f7-fd6d-414e-aef1-24e4219d6eb4 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.290893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.291990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.292508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.293009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.293911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.296129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394a25d-5e14-40a0-90bf-0ea9e7e522e7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.308369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.309600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.344198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 772/1550] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.345063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.345981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 779/1551] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:38.347681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8aed-17e3-42a4-afbe-eeac7d13ce7a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.348016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8aed-17e3-42a4-afbe-eeac7d13ce7a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.348342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8aed-17e3-42a4-afbe-eeac7d13ce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.357142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaa098-1f8c-40fd-8c17-2a2c9ab72ef1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.376195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ba6b8aed-17e3-42a4-afbe-eeac7d13ce7a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.377396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ba6b8aed-17e3-42a4-afbe-eeac7d13ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.428738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.429610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.429898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.430178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.430904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.432591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822e05-2223-4766-90a5-d7f6c1c01a41 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.442035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 780/1552] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.443442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.443737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.444013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.447459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 773/1553] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:38.448981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2c6e3-b629-4eae-a79f-db3c692e1e90 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.449277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2c6e3-b629-4eae-a79f-db3c692e1e90 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.449555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2c6e3-b629-4eae-a79f-db3c692e1e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.463731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.464924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.467167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d9f2c6e3-b629-4eae-a79f-db3c692e1e90 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.468371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d9f2c6e3-b629-4eae-a79f-db3c692e1e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.472820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 781/1554] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.476079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.477634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 774/1555] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.478709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110d32-858a-49d2-860f-bcfdd1684f43 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.478996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110d32-858a-49d2-860f-bcfdd1684f43 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.479271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba110d32-858a-49d2-860f-bcfdd1684f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.495097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15c234-783d-4c4d-aee7-ec7773013524 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.497775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ba110d32-858a-49d2-860f-bcfdd1684f43 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.498969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ba110d32-858a-49d2-860f-bcfdd1684f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.499644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.500303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.500580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.500849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.501525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.502534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:38.502534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:35:38.502534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:35:38.503830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818e387-5a91-42f1-8bdd-a8b52a51c697 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.507266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 775/1556] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.508497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.508783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.509062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.516647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:38.526000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 782/1557] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/users => generated 294 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:38.527631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.527919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.528194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.529971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.531460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.540433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 776/1558] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.546181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.550782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b81ba-d31b-4206-83d2-974fc3d678b2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.552238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.554332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.556226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.556886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.557169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.557447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.558193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.558496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.560131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d4762-7b43-4392-a9ff-a018a9ffee32 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.562333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 783/1559] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.563086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.573229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 777/1560] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:38.574313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e301085-4350-4c7a-929b-a6bc58a0dd24 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.574573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66425-0c56-4309-96b3-a58116e338b6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.574865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66425-0c56-4309-96b3-a58116e338b6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.575143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66425-0c56-4309-96b3-a58116e338b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.583986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.584869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.585143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.585411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] PATH_INFO: `/v3/projects/e068f9b959004be4ad9c778d9010561a/users/ee885a29e6fa4b1980d7ae1d03b2b476/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.592711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a8f66425-0c56-4309-96b3-a58116e338b6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.593912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a8f66425-0c56-4309-96b3-a58116e338b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.597376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:38.601100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e068f9b959004be4ad9c778d9010561a, user_id=ee885a29e6fa4b1980d7ae1d03b2b476, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.603697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 778/1561] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.604549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.604968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.605255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.605531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.620249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 784/1562] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:38 2022] PUT /identity/v3/projects/e068f9b959004be4ad9c778d9010561a/users/ee885a29e6fa4b1980d7ae1d03b2b476/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:38.621011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.629050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.630291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.684608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d783e1-07d1-4f32-ab12-73635ea64947 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.689244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.689623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 779/1563] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.689903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.690186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.690462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.690569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.691246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.691585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.693605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114bcdf-bbf7-4ac6-82af-877a88562fe7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.700357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca1d79-83ff-4759-9cf6-e0b5a024a27d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.705038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.705690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.705973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.706183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 785/1564] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:38.706304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.706942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.707515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.707808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.708224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.708552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cdf563-1e62-4198-9329-e93f6e33248f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.726029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 780/1565] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:38.726029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.726029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.726029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.733194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.734705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.742701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.743216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 786/1566] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.743954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.744209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.753908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a7541-942f-49ec-b05d-1750a9143381 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.757831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 781/1567] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.757831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.758570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.759389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.759682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.759957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.760641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.762620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.764458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2740a09-8cf0-4363-b345-db384f7c19c2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.769228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.769869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.770151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.770422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.771696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.772129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.773783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f52a2-4a49-48f5-8e70-e4b4d39ed410 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.793782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 782/1568] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:38.793782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.793782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.793782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.794292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 787/1569] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:38.794292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.809560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:38.810763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:38.855735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c09e22-ac77-4c38-be9d-74199be8a79b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.860464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.861125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.861416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.861692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.862410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.862966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.864607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.869447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 783/1570] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:38.870267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.879280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882993d-a726-4538-aa85-e84161cd55fe None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.883981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.884636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.884921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.885200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.886019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.886153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 788/1571] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:38.887009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.887823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.897522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15f38-b515-4ec0-b19c-45faca9aa7b9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.902530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.903176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.903484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.903755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.904463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.906147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.918196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 784/1572] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:38.918196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.918451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 789/1573] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:38.918451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:38.928236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ba90a-c667-4d63-ac20-468dd3edf755 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.948548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:38.965422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:38.982759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979a91-13a6-4ef9-a85c-cb3302b9aa05 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:38.984379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.985041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.985325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.985603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.986324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.986649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.987426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:38.988084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:38.988278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.988396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:38.988630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:38.989346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:38.989654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:38.991476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:38.999995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 790/1574] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:39.000784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.010352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 785/1575] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:39.010352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.012819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb22ea8-d0ed-40a1-baa9-defc58c6d2ac admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.017407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.018092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.018378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.018656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.019387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.020186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c90a8a-db6f-474e-9c6f-11a0d070a241 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.021896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.025448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.026087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.026367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.026641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.027372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.029304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.046060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 791/1576] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 285 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.046060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.060264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 786/1577] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.060939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.099201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b69c87-d025-467d-8cbd-09dd1b63cede admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.124034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca98b75-023d-4310-b727-59aff75448d3 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.128748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.129413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.129684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.129963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.131366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.133410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.148923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.149945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.150215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.150600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.162713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 787/1578] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.162713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.163001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.163001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.172154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 792/1579] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:39.173004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.182032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d83e-acc6-4308-a268-8318192a94c9 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.228551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f767ae-ecd0-476d-880b-1360738226fe admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.233913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.234714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.234975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.235243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.235977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.238098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.247224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.248238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.248590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.248957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] PATH_INFO: `/v3/projects/07eb1473648d490b90a1e7debc45d57d/users/a37198e6d4424a51800655d3bd0a3ee6/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.265649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 788/1580] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.265649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.268647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07eb1473648d490b90a1e7debc45d57d, user_id=a37198e6d4424a51800655d3bd0a3ee6, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.272532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.275354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ce01-7cdb-470b-960f-1a72a7106c15 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.280691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.281261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.281538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.282103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:39.282318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.283170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.285248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.291654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 793/1581] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:39 2022] PUT /identity/v3/projects/07eb1473648d490b90a1e7debc45d57d/users/a37198e6d4424a51800655d3bd0a3ee6/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:39.297661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.298934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:39.318517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 789/1582] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.320368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.377421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292931-6790-4181-9632-2c8aede587f2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.394407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc879fe-61ea-456c-bcb7-8512d90a1aa6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.400586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.401364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.401718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.402062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.403462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.404616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.417521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 790/1583] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.418427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.428646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499671a7-c4b4-4d89-9bf3-6bea992952a9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.444752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.445448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.445735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.446017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.446714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.448662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.461949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.462966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.463252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.463634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.464257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.466559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.471270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 791/1584] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 282 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.472310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.484019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b5980-4e95-4375-ba0e-8b59aa0c28fa admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.489493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.490290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.490573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.490959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.491908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.493908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadca8-0730-4108-8a4d-cdb496c591d7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.502079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 794/1585] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/projects => generated 393 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.502079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.523862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 792/1586] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.523862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.523862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.523862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.545111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:39.546384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:39.580711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc952acf-569a-487e-b0de-97a1f17eb101 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.587764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.591606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.591606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.591606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.591606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.593422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.607325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 795/1587] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.607518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.618632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca76c2d-9899-4595-a00e-3c58c5a05c7c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.621302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 793/1588] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:39.622404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.627755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.628608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.629053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.629424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.630506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.632861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.637084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a35e8-7707-480a-b73a-2128f1e1c7f6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.645546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.648090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.648513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.649002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.650063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.651764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.663760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 794/1589] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.664681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.669669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 796/1590] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 302 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.670836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.678520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae7620-0c2e-4eac-a3cd-2f5e230349bb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.683136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.683834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.684139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.684415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.685138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.686286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3ab3-f057-4d73-ab1a-1f9753021c19 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.686725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.694072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.695240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.695777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.696287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.697462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.699924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 795/1591] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.700960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.701099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.711590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b1003-891e-404c-afc5-7e961ab842f4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.714026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:39.716464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.717156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.717492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.717769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.718493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.720328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.732160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 796/1592] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.733019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.742701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 797/1593] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:39 2022] POST /identity/v3/users => generated 297 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:39.743185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bbcb2-f1a8-4f5a-853d-ee9eba4987c4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.744345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.750043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.750822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.751107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.751414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.752140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.753733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.764866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 797/1594] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.765688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.779162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe81e-bb69-4bb9-aa79-d39fe898d4b4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.784171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.785776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.786194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.786868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] PATH_INFO: `/v3/projects/4ca30f23f61d45bbbc8f3861696bf5f2/users/8f7b5c0764bf498f8dd90e1fa9bb8a24/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.791658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:39.806951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca30f23f61d45bbbc8f3861696bf5f2, user_id=8f7b5c0764bf498f8dd90e1fa9bb8a24, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.808024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:39.808024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:35:39.808024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:35:39.809944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.813615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdd7e8-8515-4130-896e-fb4397576f6e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.819178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.819863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.820158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.820434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.825705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 798/1595] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:39 2022] PUT /identity/v3/projects/4ca30f23f61d45bbbc8f3861696bf5f2/users/8f7b5c0764bf498f8dd90e1fa9bb8a24/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:39.826388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.833422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.835665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.848552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 798/1596] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:39.850303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.860624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdce3a-fc5c-48b7-8d56-6109911dd414 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.915946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0892e-29ca-4887-b896-e0f5128059a5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.923195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.925224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.925630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.926007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.927000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.929393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.947425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.948448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.948890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.949495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.950708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.951328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 799/1597] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.953260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:39.956059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.969771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 799/1598] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:39.970760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575fcb-17fa-461d-bd56-d8d0a4ddd154 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:39.971172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:39.985989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:39.986910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:39.987477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:39.987938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:39.989300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:39.991392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.002415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458676-d5af-42e0-a223-53c2dd1bd694 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.011776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.014953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.015253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.015564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.016308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.020878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085a92b-354a-4c2d-b80a-2a3c329c8fc4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.022349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 800/1599] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:40.023554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.040052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 800/1600] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:40.041619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.041912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.042194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.061996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cc598-12d8-430b-87a7-883d9f47f54f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.074957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.077285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.077902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.078342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] PATH_INFO: `/v3/projects/e410e4c56d7347419d7784bf56294527/users/38dc28d556424f539cdc8c4b67fdb996/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.078779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] MFA Rules not processed for user `8f7b5c0764bf498f8dd90e1fa9bb8a24`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:40.080951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:40.107067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e410e4c56d7347419d7784bf56294527, user_id=38dc28d556424f539cdc8c4b67fdb996, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.113709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.141610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 801/1601] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:40 2022] PUT /identity/v3/projects/e410e4c56d7347419d7784bf56294527/users/38dc28d556424f539cdc8c4b67fdb996/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:40.156895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.237507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b669-13eb-4cf5-b496-3bf7bc6ec4ca admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.244345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.245164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.245989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.246362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.247469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.252191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 801/1602] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:40.253414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.256375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.281604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 802/1603] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:40.283270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.284929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48716327-eeba-48b5-8490-a57dedf01681 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.295992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.297151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.297613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.298058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] PATH_INFO: `/v3/projects/61343795f34e48f887b4381f7813db90/users/b35ffe4d803b4b80b7248a7a560f05c1/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.299673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318c83-577e-427b-956c-f645656ccbf0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.306173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:40.310554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.311704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.312052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.312401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] PATH_INFO: `/v3/projects/f923122fff9b4011a092bb70c4ec6ff3/users/a45d24d55fab4e9e852f2c65c771be17/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.330498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61343795f34e48f887b4381f7813db90, user_id=b35ffe4d803b4b80b7248a7a560f05c1, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.334426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8953-13a8-41a2-938a-369989721321 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.334811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f923122fff9b4011a092bb70c4ec6ff3, user_id=a45d24d55fab4e9e852f2c65c771be17, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.338464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.359511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 803/1604] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:40 2022] PUT /identity/v3/projects/f923122fff9b4011a092bb70c4ec6ff3/users/a45d24d55fab4e9e852f2c65c771be17/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:40.360057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 802/1605] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:40 2022] PUT /identity/v3/projects/61343795f34e48f887b4381f7813db90/users/b35ffe4d803b4b80b7248a7a560f05c1/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:40.361569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.362035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.362501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.364320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.409127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:40.417259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] MFA Rules not processed for user `38dc28d556424f539cdc8c4b67fdb996`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:40.418555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:40.521950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f408ad-7090-43de-aff4-7a158ae69233 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.538507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.539173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.539408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.539694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.540430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.542196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.542868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 803/1606] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:40.548210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.562216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004bf0b-01d7-4e15-81cc-2f177214c4aa None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.567534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 804/1607] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:40.567534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.583876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dd0c4-d77c-4738-ba03-e2f3a7e7670a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.585052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.586140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.586701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.587232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.588291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.589430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.590381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.590669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.590949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] PATH_INFO: `/v3/projects/053ad0f564e3458398158e358c446eb1/users/74a8469b79df41d6bd4e597a2a9d3bd7/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.591416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f60ba-5d00-4c25-bc70-f464a85ba456 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.617687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] RBAC: Authorizing `identity:create_grant(project_id=053ad0f564e3458398158e358c446eb1, user_id=74a8469b79df41d6bd4e597a2a9d3bd7, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.620530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba0ae-0204-420d-a0ae-0b98356464de admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.624829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 804/1608] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:40.627441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.627954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.628564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.650240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:40.651970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:40.652561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 805/1609] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:40 2022] PUT /identity/v3/projects/053ad0f564e3458398158e358c446eb1/users/74a8469b79df41d6bd4e597a2a9d3bd7/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:40.657071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.657071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.657071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.717707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] MFA Rules not processed for user `b35ffe4d803b4b80b7248a7a560f05c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:40.719497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:40.785570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 805/1610] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:40.787347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.799990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995d8ea-fe33-489a-af76-1309b9c9b30f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.807248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.809714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.810191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.810655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:40.811548 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:40.814674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:40.856685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 806/1611] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:40.858755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.874541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f383d1-cd4d-48b0-8cab-c2d2b7eb5d5f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:40.877374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 806/1612] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:40.884006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:40.959200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:40.985701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:40.987802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:40.989281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:40.990723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] PATH_INFO: `/v3/projects/13b4839566df4751bfde1f4975cdcb78/users/bc74ebac003a4f828fdb44db6dc42887/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.010001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b4839566df4751bfde1f4975cdcb78, user_id=bc74ebac003a4f828fdb44db6dc42887, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.015280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa2c1f-6bdd-4d92-9389-152160d2511c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.019531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba9bc8-c342-4ab2-a2a2-a56e651759cc admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.030143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.031302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.032095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.032636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.035570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.039913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.042093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 807/1613] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:40 2022] PUT /identity/v3/projects/13b4839566df4751bfde1f4975cdcb78/users/bc74ebac003a4f828fdb44db6dc42887/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:41.045994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.045994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.045994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.051704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:41.081358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 807/1614] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:40 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:41.082871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.106018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] MFA Rules not processed for user `74a8469b79df41d6bd4e597a2a9d3bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:41.108632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:41.230136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f907c8e-4f1f-4bd1-b541-4e35c6dba809 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.247467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 808/1615] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:41.250005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.346558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.347405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.347746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.348349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.389934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bab76-d015-44b0-939a-56c2a39503ae None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.400975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.401769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.402127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.402482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.403343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.405375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.448339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.453632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933538-a127-42ac-a419-32d72f2b7efc service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.468843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 809/1616] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/projects => generated 365 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:41.468843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.483788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 808/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:41.485543 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.485987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.486536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.527366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:41.531155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:41.581689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:41.607450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7d0f5-21e2-41a9-bedf-c337be108a3e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.616143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.617031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.617319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.617596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.618301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.620295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.698160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 810/1618] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:41.698160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.717709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e03d2c-2af7-4d87-a917-9b1be721a03a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.726785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.727921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.728522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.729041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 809/1619] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:41.729577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.729960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.730795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.732753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.734647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:41.757166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700541d-9a3d-41e1-9bac-493126de342a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.762510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 811/1620] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:41.763775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.774033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.774736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.775018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.775793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.776040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.778002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.799021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94060d3f-de17-430e-b9de-8a1f787cd5ff admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.810716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.811522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.811813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.812094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.812804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.815681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.847750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 810/1621] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:41.847750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.865302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 812/1622] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:41 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:41.866387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.877090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d19ab-7d47-4b4d-b4a8-4440cfa717e5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.964145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f7b6f-753d-4eba-a304-43465ef8ac45 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:41.971107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.972677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.973437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.973975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:41.975385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:41.977493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:41.989045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:41.990590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:41.994776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 811/1623] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:41.995653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:41.995765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:41.995867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] PATH_INFO: `/v3/projects/a3c36e9895b0479a9bac7ae44379715b/users/a2b4402e16114fd8988817b8a3f04a41/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.033704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c36e9895b0479a9bac7ae44379715b, user_id=a2b4402e16114fd8988817b8a3f04a41, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.043189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.069184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44358c7-2beb-4087-978f-da7d64caac25 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.073194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 813/1624] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:41 2022] PUT /identity/v3/projects/a3c36e9895b0479a9bac7ae44379715b/users/a2b4402e16114fd8988817b8a3f04a41/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:42.076506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.188864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.189684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.190072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.190605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.205624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:42.244121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd28a9f-30f3-430b-8c6a-367025abddfd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.257376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.258298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.258585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.260128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.262059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.264098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.284898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.287859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.303084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 812/1625] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:42.306642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 814/1626] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:42 2022] POST /identity/v3/users => generated 308 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:42.307517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.308461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.330444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edd6da-e423-4afb-a910-9e988e6d713d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.333397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c0e2c-6041-4b2c-9b12-4462bcfaf9f8 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.343631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.344774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.345061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.345444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] PATH_INFO: `/v3/projects/615933bca2cc471d86223ce5a686fa5a/users/a5508783f8494fedbbf072cbfed355da/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.351308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.353620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.354160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.354591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.362666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.365528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.382030 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 813/1627] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:42.383095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.385445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=615933bca2cc471d86223ce5a686fa5a, user_id=a5508783f8494fedbbf072cbfed355da, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.388251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.395408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd1fb-7884-401d-a112-4153e662a91c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.402797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.404012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.404416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.404741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.405703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.408003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.411102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 815/1628] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:42 2022] PUT /identity/v3/projects/615933bca2cc471d86223ce5a686fa5a/users/a5508783f8494fedbbf072cbfed355da/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:42.412322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.422311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 814/1629] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:42.423157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.450758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695eda6-7635-4185-b220-5288d4e4c759 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.489267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:42.556079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.557617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.558293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.558680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] PATH_INFO: `/v3/projects/7a3b0050942544b897aa9b8a78b4284a/users/e1a6172975e547a49834d40dacf50f65/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.575655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158477-af59-4f8c-be9f-85d014ea4d2b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.584693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.585785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.586221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.586673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.587617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.589524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.593369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3b0050942544b897aa9b8a78b4284a, user_id=e1a6172975e547a49834d40dacf50f65, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.603147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.608852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 816/1630] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:42.610097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.622900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003795-8849-469d-8240-00135ab46570 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.646755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 815/1631] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:42 2022] PUT /identity/v3/projects/7a3b0050942544b897aa9b8a78b4284a/users/e1a6172975e547a49834d40dacf50f65/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:42.650823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.741918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.742610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.742894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.743172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.746468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.748178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.783912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 817/1632] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:42.787389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.811058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9af3f-553e-4d61-aaf2-a8718660ac91 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.823009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cabc4-35a7-456f-9298-a17c374de735 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:42.827041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.828002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.829419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.830042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] PATH_INFO: `/v3/projects/a3c36e9895b0479a9bac7ae44379715b/users/a2b4402e16114fd8988817b8a3f04a41/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.831647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:42.833400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:42.833961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:42.834583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] PATH_INFO: `/v3/projects/ae2052ec8eb04059b9a5a3315c52d248/users/3c3aa66b05f94f8ea86b88039afa1a7e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:42.834958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c36e9895b0479a9bac7ae44379715b, user_id=a2b4402e16114fd8988817b8a3f04a41, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.837637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.885814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 816/1633] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:42 2022] PUT /identity/v3/projects/a3c36e9895b0479a9bac7ae44379715b/users/a2b4402e16114fd8988817b8a3f04a41/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:42.885814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:42.890378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:42.896880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2052ec8eb04059b9a5a3315c52d248, user_id=3c3aa66b05f94f8ea86b88039afa1a7e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:42.903031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:42.939164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 818/1634] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:42 2022] PUT /identity/v3/projects/ae2052ec8eb04059b9a5a3315c52d248/users/3c3aa66b05f94f8ea86b88039afa1a7e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:42.939164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.032940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fb2ec-7946-454a-84df-968e7d5a3236 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.076247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b11f01-ab68-4633-bd1a-f9de2d5c00bd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.088083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.089075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.089511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.090989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] PATH_INFO: `/v3/projects/615933bca2cc471d86223ce5a686fa5a/users/a5508783f8494fedbbf072cbfed355da/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.100433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] RBAC: Authorizing `identity:create_grant(project_id=615933bca2cc471d86223ce5a686fa5a, user_id=a5508783f8494fedbbf072cbfed355da, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.105407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29d49c-acca-4a61-ac5b-0f9a2c17097b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.159046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 819/1635] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:42 2022] PUT /identity/v3/projects/615933bca2cc471d86223ce5a686fa5a/users/a5508783f8494fedbbf072cbfed355da/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:43.164064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef414-a468-4d7b-a0d3-e78696761fae None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.164437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef414-a468-4d7b-a0d3-e78696761fae None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.165295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef414-a468-4d7b-a0d3-e78696761fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.189769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.190464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.190742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.191056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.191819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.194278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.235045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 817/1636] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:43.240702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.251748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b33ef414-a468-4d7b-a0d3-e78696761fae None None] MFA Rules not processed for user `a2b4402e16114fd8988817b8a3f04a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:43.253358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b33ef414-a468-4d7b-a0d3-e78696761fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:43.277951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3c1e-631f-4426-849c-79dd9e0f7a31 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.292024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:43.428904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 820/1637] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:43.430933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.431351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.441898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.467541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.467541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.467541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.468407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.468972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.470735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.485846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] MFA Rules not processed for user `a5508783f8494fedbbf072cbfed355da`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:43.487422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:43.488625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 818/1638] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:43.489470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.513556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabc422-19a3-40e5-9272-14c9835ccc84 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.531220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.532154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.532434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.532705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] PATH_INFO: `/v3/projects/7a3b0050942544b897aa9b8a78b4284a/users/e1a6172975e547a49834d40dacf50f65/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.537822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a3b0050942544b897aa9b8a78b4284a, user_id=e1a6172975e547a49834d40dacf50f65, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.541303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.572648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 819/1639] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:43 2022] PUT /identity/v3/projects/7a3b0050942544b897aa9b8a78b4284a/users/e1a6172975e547a49834d40dacf50f65/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:43.573869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.658493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:43.711339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 821/1640] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:43.712003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.734729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8281ae-eca7-459a-9cbc-fb8d09a89f4d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.742252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.742931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.743214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.745865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.747538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03999935-b027-43d9-9f15-34f682ec28a5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.878747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:43.879958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.880097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.880407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] PATH_INFO: `/v3/projects/ae2052ec8eb04059b9a5a3315c52d248/users/3c3aa66b05f94f8ea86b88039afa1a7e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.884473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2052ec8eb04059b9a5a3315c52d248, user_id=3c3aa66b05f94f8ea86b88039afa1a7e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.887887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c97338-419a-4077-bf23-c849b33b1b1d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.907942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:43.909619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:43.917675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 822/1641] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:43 2022] PUT /identity/v3/projects/ae2052ec8eb04059b9a5a3315c52d248/users/3c3aa66b05f94f8ea86b88039afa1a7e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:43.918827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5740-af51-4bc1-a000-aaf99b4d91d7 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:43.920358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5740-af51-4bc1-a000-aaf99b4d91d7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:43.920652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a5740-af51-4bc1-a000-aaf99b4d91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:43.934579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 820/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:43.935570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:43.945254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b43e-384a-44eb-aab2-d8affce2b0e0 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:43.973352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-937a5740-af51-4bc1-a000-aaf99b4d91d7 None None] MFA Rules not processed for user `e1a6172975e547a49834d40dacf50f65`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:43.976143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-937a5740-af51-4bc1-a000-aaf99b4d91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:44.060983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:44.061199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:44.065722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:44.066081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:44.066382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:44.233808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 823/1643] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:44.238263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:44.238696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:44.239310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:44.240003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:44.240841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:44.296938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 821/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:44.297983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:44.317209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459781-0b2f-4652-b072-30d512d97d06 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:44.319899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] MFA Rules not processed for user `3c3aa66b05f94f8ea86b88039afa1a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:44.323792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:44.336444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:44.340343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:44.340574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:44.343413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:44.417555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:44.492541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:44.494074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:44.517167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 822/1645] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:44.518001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:44.542947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a698-2e28-45f3-b7d2-9e054eddb1c6 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:44.552023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:44.553261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:44.553577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:44.553919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:44.576862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:44.578463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:44.596188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 824/1646] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:44 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:44.620989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 823/1647] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:44.786236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:44.799697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc393cba-0684-4a44-ac40-12fa7f6aa8f9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:44.806767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:44.808576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:44.811342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:44.811864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:44.838447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:44.840048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:44.860914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 825/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:44 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:45.038703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:45.050731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9714-250b-4639-9ea0-3ba4c0ce82bb service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:45.058455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:45.059160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:45.064360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:45.064931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:45.185893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:45.187278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:45.225268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 824/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:45.559033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:45.576581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36227ca-887d-4815-bf69-a991a66fbe6b service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:45.595576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:45.600848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:45.608192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:45.610651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:45.787644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:45.789700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f161f-76fe-48fd-8ddf-cd48dc95259b service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:45.806247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 826/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:46.325123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:46.508204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d8672-58b3-4c5f-b386-050db3b34ec2 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:46.518683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:46.527724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:46.528009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:46.529803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] PATH_INFO: `/v3/users/74a8469b79df41d6bd4e597a2a9d3bd7` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:46.532323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a8469b79df41d6bd4e597a2a9d3bd7)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:46.545129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:46.638130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a8469b79df41d6bd4e597a2a9d3bd7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:46.651588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a8469b79df41d6bd4e597a2a9d3bd7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:46.653687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:46.678881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a8469b79df41d6bd4e597a2a9d3bd7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:46.679967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a8469b79df41d6bd4e597a2a9d3bd7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:46.705263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9f41b675-077a-40f8-b153-cd8f3f4c09d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a8469b79df41d6bd4e597a2a9d3bd7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:46.727612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 825/1651] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:46 2022] DELETE /identity/v3/users/74a8469b79df41d6bd4e597a2a9d3bd7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:46.732724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:46.733582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:46.733996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:46.796486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:46.797726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:46.913259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:47.051943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 827/1652] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:47.118895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762327e-3604-4bea-9b4b-e9548278d78b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:47.119435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762327e-3604-4bea-9b4b-e9548278d78b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:47.119855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762327e-3604-4bea-9b4b-e9548278d78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:47.121628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:47.123526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:47.179293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f762327e-3604-4bea-9b4b-e9548278d78b None None] MFA Rules not processed for user `ee885a29e6fa4b1980d7ae1d03b2b476`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:47.180712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f762327e-3604-4bea-9b4b-e9548278d78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:47.310724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411aefe-bffc-4f4d-9566-7ce32cd4694f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:47.322922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:47.323625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:47.323925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:47.324192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:47.352112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:47.353504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaf3da-1986-45c2-a71b-6b800d1061ac service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:47.370790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 828/1653] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:47.392414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 826/1654] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:47.443615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe9cb-f943-4027-8b8e-50a66b6c79e5 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:47.443909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe9cb-f943-4027-8b8e-50a66b6c79e5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:47.444190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbe9cb-f943-4027-8b8e-50a66b6c79e5 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:47.445604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 829/1655] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jul 20 09:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:35:47.465007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:47.465813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:47.466466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:47.596319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:47.597786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:47.817165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 827/1656] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jul 20 09:35:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:47.839534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbe9cb-f943-4027-8b8e-50a66b6c79e5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:47.863034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbe9cb-f943-4027-8b8e-50a66b6c79e5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:47.875575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:47.878182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:47.881518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:47.881996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:47.901096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:47.903048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:47.924175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 830/1657] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:49.425305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:49.429084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:49.457529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab1cd7-f3f2-43cb-814f-2ab9870a7565 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:49.474683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:49.476019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:49.476516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:49.476981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] PATH_INFO: `/v3/users/b35ffe4d803b4b80b7248a7a560f05c1` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:49.480113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35ffe4d803b4b80b7248a7a560f05c1)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:49.484117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:49.502482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:49.508595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:49.560101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb8a38-45ed-46a0-b6d9-04e86133d248 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:49.578513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:49.582987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:49.584157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:49.586479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] PATH_INFO: `/v3/projects/053ad0f564e3458398158e358c446eb1` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:49.592976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053ad0f564e3458398158e358c446eb1)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:49.600204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:49.604011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:49.635484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35ffe4d803b4b80b7248a7a560f05c1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.656600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35ffe4d803b4b80b7248a7a560f05c1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.701059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35ffe4d803b4b80b7248a7a560f05c1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.702520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35ffe4d803b4b80b7248a7a560f05c1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.736585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23aa4cee-f63e-42d1-8f69-3e77408c1fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35ffe4d803b4b80b7248a7a560f05c1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.755228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053ad0f564e3458398158e358c446eb1', 'request_id': 'req-7890e801-c860-4dc2-878d-1aefc38007f3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.757745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 828/1658] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:49 2022] DELETE /identity/v3/users/b35ffe4d803b4b80b7248a7a560f05c1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:49.764919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:49.765210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:49.765562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:49.767237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053ad0f564e3458398158e358c446eb1', 'request_id': 'req-7890e801-c860-4dc2-878d-1aefc38007f3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.787877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053ad0f564e3458398158e358c446eb1', 'request_id': 'req-7890e801-c860-4dc2-878d-1aefc38007f3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.811945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:49.813874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:49.834475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] The token cache is being invalidate because project 053ad0f564e3458398158e358c446eb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:49.834844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.837009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:49.842287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 831/1659] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:49 2022] DELETE /identity/v3/projects/053ad0f564e3458398158e358c446eb1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:49.846683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:49.851951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:49.972031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:50.048669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 829/1660] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:50.063926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890e801-c860-4dc2-878d-1aefc38007f3 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:50.072231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:50.073162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:50.073755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:50.074606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:50.075045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:50.075579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] PATH_INFO: `/v3/users/3c3aa66b05f94f8ea86b88039afa1a7e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:50.078380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c3aa66b05f94f8ea86b88039afa1a7e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:50.079767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: WARNING py.warnings [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:50.079767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: warnings.warn(msg) Jul 20 09:35:50.079767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: Jul 20 09:35:50.082272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:50.102962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061c16a-61ad-4ae0-b1c1-7fe111d14952 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:50.119665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:50.120464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:50.120773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:50.126059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] PATH_INFO: `/v3/users/8f7b5c0764bf498f8dd90e1fa9bb8a24` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:50.136108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f7b5c0764bf498f8dd90e1fa9bb8a24)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:50.142029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:50.259236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c3aa66b05f94f8ea86b88039afa1a7e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.271195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f7b5c0764bf498f8dd90e1fa9bb8a24'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.283033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c3aa66b05f94f8ea86b88039afa1a7e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.284690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c3aa66b05f94f8ea86b88039afa1a7e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.290851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f7b5c0764bf498f8dd90e1fa9bb8a24'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.318107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f7b5c0764bf498f8dd90e1fa9bb8a24'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.318900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f7b5c0764bf498f8dd90e1fa9bb8a24'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.328946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f7b5c0764bf498f8dd90e1fa9bb8a24'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.342856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 830/1661] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:50 2022] DELETE /identity/v3/users/8f7b5c0764bf498f8dd90e1fa9bb8a24 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:50.343039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c3aa66b05f94f8ea86b88039afa1a7e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.343764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:50.347576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:50.359504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ab9576a3-31b3-43bd-8a7d-fa6fec5b7bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c3aa66b05f94f8ea86b88039afa1a7e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:50.372265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 832/1662] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:49 2022] DELETE /identity/v3/users/3c3aa66b05f94f8ea86b88039afa1a7e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:50.374382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:50.374804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:50.375089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:50.409282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:50.413685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:50.416904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:50.486368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c661d-bcf0-49ce-8761-76300f4868f7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:50.562436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:50.564297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:50.564802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:50.565568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:50.569319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:50.586061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 833/1663] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:50.591734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:50.595541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26606c2d-83be-4c02-9382-19d76c4634e4 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:50.610361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 831/1664] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:50.643086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:50.644608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:50.667832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50363ae-7669-4c58-9d61-d81f0bb96e7d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:50.678811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:50.679524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:50.679826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:50.680085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:50.708211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:50.710115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f447b32-e400-4fc5-83b7-3c5adc161022 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:50.731721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 834/1665] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:51.937961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee4c91-63ec-435d-a9e6-10dd9cc40615 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:51.938602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee4c91-63ec-435d-a9e6-10dd9cc40615 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:51.938602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee4c91-63ec-435d-a9e6-10dd9cc40615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:51.983605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0bee4c91-63ec-435d-a9e6-10dd9cc40615 None None] MFA Rules not processed for user `a37198e6d4424a51800655d3bd0a3ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:51.986178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0bee4c91-63ec-435d-a9e6-10dd9cc40615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:52.135885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 832/1666] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:52.181964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:52.182492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:52.182717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:52.187911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 835/1667] 10.222.0.18 () {54 vars in 962 bytes} [Wed Jul 20 09:35:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:35:52.210096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:52.210422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:52.210660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:52.261817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:52.263078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:52.509719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 833/1668] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Jul 20 09:35:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:52.522760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:52.527988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:52.534304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:52.542256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:52.551362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc635b6-182b-42e3-adce-e13b8abd07f6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:52.572983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:52.577909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:52.578255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:52.578547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:52.586991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3317-d465-46af-bbe4-b550dedcc340 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:52.609525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:52.615273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:52.615598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:52.616460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:52.618294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:52.618591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] PATH_INFO: `/v3/projects/4ca30f23f61d45bbbc8f3861696bf5f2` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:52.630037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca30f23f61d45bbbc8f3861696bf5f2)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:52.666915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:52.678359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 834/1669] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:52.779521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:52.780981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:52.814370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23113cbe-0a24-47d5-bc43-63e8151c621b service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:52.843014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:52.843936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:52.844223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:52.844506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] PATH_INFO: `/v3/projects/ae2052ec8eb04059b9a5a3315c52d248` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:52.858428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2052ec8eb04059b9a5a3315c52d248)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:52.859637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: WARNING py.warnings [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:35:52.859637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: warnings.warn(msg) Jul 20 09:35:52.859637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: Jul 20 09:35:52.860797 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:52.877034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:52.889772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca30f23f61d45bbbc8f3861696bf5f2', 'request_id': 'req-b5232991-6ac7-4b0e-8241-ba3ffef45da3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:52.906173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca30f23f61d45bbbc8f3861696bf5f2', 'request_id': 'req-b5232991-6ac7-4b0e-8241-ba3ffef45da3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:52.924223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca30f23f61d45bbbc8f3861696bf5f2', 'request_id': 'req-b5232991-6ac7-4b0e-8241-ba3ffef45da3'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:52.948942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] The token cache is being invalidate because project 4ca30f23f61d45bbbc8f3861696bf5f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:52.949229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:52.951095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:52.956048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 836/1670] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:52 2022] DELETE /identity/v3/projects/4ca30f23f61d45bbbc8f3861696bf5f2 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:52.959878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:52.962538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:53.166450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5232991-6ac7-4b0e-8241-ba3ffef45da3 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:53.169352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2052ec8eb04059b9a5a3315c52d248', 'request_id': 'req-ad8f4fdc-60a3-44fb-805c-65e137c547ec'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.218691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2052ec8eb04059b9a5a3315c52d248', 'request_id': 'req-ad8f4fdc-60a3-44fb-805c-65e137c547ec'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.240992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2052ec8eb04059b9a5a3315c52d248', 'request_id': 'req-ad8f4fdc-60a3-44fb-805c-65e137c547ec'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.326428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] The token cache is being invalidate because project ae2052ec8eb04059b9a5a3315c52d248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:53.326970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.328088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.333617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 835/1671] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:52 2022] DELETE /identity/v3/projects/ae2052ec8eb04059b9a5a3315c52d248 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:35:53.340455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:53.346915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:53.366698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:53.368837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.369893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.370783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] PATH_INFO: `/v3/users/a2b4402e16114fd8988817b8a3f04a41` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.374088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b4402e16114fd8988817b8a3f04a41)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:53.387936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:53.416341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f4fdc-60a3-44fb-805c-65e137c547ec admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:53.449635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:53.450530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.450813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.451091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] PATH_INFO: `/v3/projects/61343795f34e48f887b4381f7813db90` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.452338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61343795f34e48f887b4381f7813db90)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:53.455901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:53.511259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b4402e16114fd8988817b8a3f04a41'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.531361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b4402e16114fd8988817b8a3f04a41'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.532455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b4402e16114fd8988817b8a3f04a41'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.562574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b4402e16114fd8988817b8a3f04a41'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.583023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-92a01a68-a745-4e65-9bdc-528bb18e76b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b4402e16114fd8988817b8a3f04a41'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.588592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:53.595345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61343795f34e48f887b4381f7813db90', 'request_id': 'req-d1ae8050-9ecb-4693-9837-485b8e5eca96'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.597172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 837/1672] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:52 2022] DELETE /identity/v3/users/a2b4402e16114fd8988817b8a3f04a41 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:53.600966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a1344-2de1-4bbf-9008-cb27cf0862a0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.601363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a1344-2de1-4bbf-9008-cb27cf0862a0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.601751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a1344-2de1-4bbf-9008-cb27cf0862a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.613754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61343795f34e48f887b4381f7813db90', 'request_id': 'req-d1ae8050-9ecb-4693-9837-485b8e5eca96'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.632125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61343795f34e48f887b4381f7813db90', 'request_id': 'req-d1ae8050-9ecb-4693-9837-485b8e5eca96'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.655010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ca9a1344-2de1-4bbf-9008-cb27cf0862a0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:53.658443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ca9a1344-2de1-4bbf-9008-cb27cf0862a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:53.668343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] The token cache is being invalidate because project 61343795f34e48f887b4381f7813db90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:53.669256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.683842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:53.692552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 836/1673] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:53 2022] DELETE /identity/v3/projects/61343795f34e48f887b4381f7813db90 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:53.696308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:53.699685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:53.809924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 838/1674] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:53.816507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.817038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.817595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.888091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:53.889335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:53.907759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 839/1675] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:53.921198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:53.925411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:53.935337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae8050-9ecb-4693-9837-485b8e5eca96 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:53.940866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eded00-dbb2-4986-b463-fed61f5d9088 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:53.947893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:53.948600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:53.950266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.950691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:53.951051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.952812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] PATH_INFO: `/v3/users/e1a6172975e547a49834d40dacf50f65` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.955564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a6172975e547a49834d40dacf50f65)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:53.960154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:53.960682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:53.961693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:53.962912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:53.966980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:53.967945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:53.987338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 840/1676] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:54.003616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.031634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a6d1f-e493-498b-9912-eeeb6ff7e2a1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.045578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.048363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.049054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.051565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.053326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.055799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.104392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 841/1677] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:54.108327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.125172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a6172975e547a49834d40dacf50f65'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:54.142582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a6172975e547a49834d40dacf50f65'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:54.164793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a6172975e547a49834d40dacf50f65'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:54.167082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a6172975e547a49834d40dacf50f65'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:54.180775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a6172975e547a49834d40dacf50f65'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:54.201480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 837/1678] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:53 2022] DELETE /identity/v3/users/e1a6172975e547a49834d40dacf50f65 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:54.210619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ed2a-6930-4e83-9149-c94396d4f422 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.297690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.298391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.298855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.299138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.299880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.301817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.312297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:54.330723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 842/1679] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:54.334470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.336100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:54.356402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66356ef0-780c-45a0-89d2-509924a7ee5b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.366834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.374291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.374765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.375152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.376114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.377926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.400724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 838/1680] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:54.404654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.421045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9383cd-7b46-4bbb-8369-4b4664e30e67 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.436683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.436683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.436683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.436683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.438815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.447887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f88664-27c8-4806-b903-823fb631a9dd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.478943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 843/1681] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:54.480111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.494624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920ed6f-558f-475e-86b8-40af77cdcea0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.508867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.511461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.512382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.512778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] PATH_INFO: `/v3/projects/831c3dfee4de496981ccaf8cb0f28c7e/users/58103a8013204ddf95b468a3ae69ef9f/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.556573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] RBAC: Authorizing `identity:create_grant(project_id=831c3dfee4de496981ccaf8cb0f28c7e, user_id=58103a8013204ddf95b468a3ae69ef9f, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.559481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.584467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 839/1682] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:54 2022] PUT /identity/v3/projects/831c3dfee4de496981ccaf8cb0f28c7e/users/58103a8013204ddf95b468a3ae69ef9f/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:35:54.595734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6f77e-16ab-420d-97fe-95ca11df3ed7 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.596660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6f77e-16ab-420d-97fe-95ca11df3ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.596943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6f77e-16ab-420d-97fe-95ca11df3ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.628874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fcb6f77e-16ab-420d-97fe-95ca11df3ed7 None None] MFA Rules not processed for user `58103a8013204ddf95b468a3ae69ef9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:54.629872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fcb6f77e-16ab-420d-97fe-95ca11df3ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:54.723915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 844/1683] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:54.729008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:54.855564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac4cb-1850-4125-a87b-39622dd8b345 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.855968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac4cb-1850-4125-a87b-39622dd8b345 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.856372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac4cb-1850-4125-a87b-39622dd8b345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.863076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a022f-1df4-49de-85d1-f0b2f38a800c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:54.874559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:54.875459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:54.875751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:54.876161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:54.877047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:54.879164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:54.885873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0dcac4cb-1850-4125-a87b-39622dd8b345 None None] MFA Rules not processed for user `ee885a29e6fa4b1980d7ae1d03b2b476`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:54.888473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0dcac4cb-1850-4125-a87b-39622dd8b345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:54.941598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 840/1684] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/projects => generated 371 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:54.946025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.015861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:55.022702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 845/1685] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:55.160361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b06d54-eea6-4d5b-943b-3e578d85761d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.172796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b01f6-1a10-41ed-bea3-4bc572d56a55 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.173682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b01f6-1a10-41ed-bea3-4bc572d56a55 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.173682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b01f6-1a10-41ed-bea3-4bc572d56a55 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.174431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:55.175445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 846/1686] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jul 20 09:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:35:55.177006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.178912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.179379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.180606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.181004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.181340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:55.181797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.187748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:55.228486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:55.229766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:55.264044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 841/1687] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:54 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:55.274070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.307326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1be1f-c694-4c3a-9f3e-adf660acb782 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.338400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:55.341589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.342510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.343085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.345395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:55.351829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:55.388890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 842/1688] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:55.392145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.423804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5915fc-7865-4562-8d3b-af0517369f18 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.449868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:55.452046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 847/1689] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:35:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:55.453156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.453909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.454531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.455020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] PATH_INFO: `/v3/projects/910cb01c470a4af580f7d7e601f200bd/users/90c7b7021982437b8095f01292d2f43a/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.455521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:55.500765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871f92d-2bf4-4a44-b143-328ee040a8cc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.512233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910cb01c470a4af580f7d7e601f200bd, user_id=90c7b7021982437b8095f01292d2f43a, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:55.517106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:55.518169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:55.518563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:55.518954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] PATH_INFO: `/v3/projects/a3c36e9895b0479a9bac7ae44379715b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:55.522688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c36e9895b0479a9bac7ae44379715b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:55.524901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:55.526721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:55.532518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:55.565560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 843/1690] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:55 2022] PUT /identity/v3/projects/910cb01c470a4af580f7d7e601f200bd/users/90c7b7021982437b8095f01292d2f43a/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:55.571332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.575426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:55.647659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c36e9895b0479a9bac7ae44379715b', 'request_id': 'req-3f360739-0791-479c-a31b-f22280e28d4f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:55.664543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c36e9895b0479a9bac7ae44379715b', 'request_id': 'req-3f360739-0791-479c-a31b-f22280e28d4f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:55.682620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c36e9895b0479a9bac7ae44379715b', 'request_id': 'req-3f360739-0791-479c-a31b-f22280e28d4f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:55.732695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] The token cache is being invalidate because project a3c36e9895b0479a9bac7ae44379715b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:55.733167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:55.742725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:55.747865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 848/1691] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:55 2022] DELETE /identity/v3/projects/a3c36e9895b0479a9bac7ae44379715b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:55.752212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:55.757405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:55.830946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cca3-9cf1-45e6-b2e9-a69c5f8a42e2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.991411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f360739-0791-479c-a31b-f22280e28d4f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:55.998175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:56.000890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.002405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.003691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.009184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:56.012203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:56.014266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.016907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.017190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.019213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:56.024886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bdd82c-5bed-4820-a077-86843ab4d22a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:56.074701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 849/1692] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:56.078361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.078570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.081603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.113303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:56.165695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:56.175759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:56.213566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 850/1693] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:56.215613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:56.250931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44745c-a1e0-43d7-8071-81d2addad4de None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:56.274368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:56.276273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.276562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.276890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] PATH_INFO: `/v3/projects/910cb01c470a4af580f7d7e601f200bd/users/90c7b7021982437b8095f01292d2f43a/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.279659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910cb01c470a4af580f7d7e601f200bd, user_id=90c7b7021982437b8095f01292d2f43a, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:56.288625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:56.290153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a677490-6235-43a6-9633-4837d748d3c3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:56.297255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e00e98-181f-4e4d-9214-16b423a350f3 service glance] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:56.333120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 851/1694] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:56 2022] PUT /identity/v3/projects/910cb01c470a4af580f7d7e601f200bd/users/90c7b7021982437b8095f01292d2f43a/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:56.334745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.335043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.335342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.340173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 844/1695] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 774 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:56.361510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.361622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.363233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.461209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:56.462567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:56.506518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] MFA Rules not processed for user `90c7b7021982437b8095f01292d2f43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:56.508897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:56.663848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 852/1696] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:56.665376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:56.678127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259cedd-da20-4bd8-9acd-187aaf0ac83a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:56.691193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:56.692021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.692468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.693502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.694383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:56.694796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:56.699757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:56.708818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:56.726164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 853/1697] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:35:56.730225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:56.760275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323b898-0b47-4ee6-9398-7befaa7e86b1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:56.766926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 845/1698] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:56.777622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:56.778357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:56.779054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:56.779165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:56.779617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:56.780019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:56.780829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:56.789112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:56.856320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 854/1699] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:56 2022] POST /identity/v3/projects => generated 407 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:35:56.858975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:56.907475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8ad15-0557-4033-bd3b-75b41d8a0a9d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.033718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a09b8-73c7-4f82-8f35-f4686c15abf4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.047368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.047594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.047787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.048196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.065815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.067552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.070076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.070491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.070729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:57.072860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.082966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.129899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 855/1700] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:56 2022] POST /identity/v3/users => generated 304 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:57.131633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:57.159564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25502-9bfb-4410-835a-0e9cde8be59d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.177714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.179044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.180856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.181267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.183153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.185402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.189165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.191785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.225373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 856/1701] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:57.227266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 846/1702] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:57.228423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:57.249909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107088a6-d807-48a9-a9af-a03796a8d2b6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.267582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.268531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.268882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.269226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.270035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.274502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.324960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 857/1703] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:57.329144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:57.346518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26cafb-6dce-4fe4-96c8-e14eae1d3ad8 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.360735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:57.363154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:57.366123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.367045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.367405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.367599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] PATH_INFO: `/v3/projects/4479dc9a15cc422fa0d4120ec03d9785/users/9299822ff7074363bfa5740c1e5e6255/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.396979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e7fbc-39fd-44f9-935a-7bdba2ed776f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:57.428728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:57.430419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.430717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.430978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] PATH_INFO: `/v3/projects/7a3b0050942544b897aa9b8a78b4284a` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.437423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a3b0050942544b897aa9b8a78b4284a)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.438949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4479dc9a15cc422fa0d4120ec03d9785, user_id=9299822ff7074363bfa5740c1e5e6255, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:57.441613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007fed8-16a7-4868-9c8d-22b1d9447d6e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.446533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:57.504432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 847/1704] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:57 2022] PUT /identity/v3/projects/4479dc9a15cc422fa0d4120ec03d9785/users/9299822ff7074363bfa5740c1e5e6255/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:57.513452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da5f1-2604-4bae-932b-d3e444f594d0 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.513945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da5f1-2604-4bae-932b-d3e444f594d0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.514556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da5f1-2604-4bae-932b-d3e444f594d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.620547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4a7da5f1-2604-4bae-932b-d3e444f594d0 None None] MFA Rules not processed for user `9299822ff7074363bfa5740c1e5e6255`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:57.622896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4a7da5f1-2604-4bae-932b-d3e444f594d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:57.629246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a3b0050942544b897aa9b8a78b4284a', 'request_id': 'req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:57.654592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a3b0050942544b897aa9b8a78b4284a', 'request_id': 'req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:57.698301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a3b0050942544b897aa9b8a78b4284a', 'request_id': 'req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:57.727205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] The token cache is being invalidate because project 7a3b0050942544b897aa9b8a78b4284a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:35:57.727555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:57.728401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c5fa4822-22fb-49ac-b1a4-4e93b5fae6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:57.730406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 858/1705] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:35:57 2022] DELETE /identity/v3/projects/7a3b0050942544b897aa9b8a78b4284a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:57.734876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.736623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.736843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.800349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:57.801666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:57.857522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 848/1706] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:35:57.858956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15b69-d3e0-4bec-ae1d-4bacaaa23269 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:57.859574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15b69-d3e0-4bec-ae1d-4bacaaa23269 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:57.859956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec15b69-d3e0-4bec-ae1d-4bacaaa23269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:57.861528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:57.939951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7ec15b69-d3e0-4bec-ae1d-4bacaaa23269 None None] MFA Rules not processed for user `a37198e6d4424a51800655d3bd0a3ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:57.941104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7ec15b69-d3e0-4bec-ae1d-4bacaaa23269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.016591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 859/1707] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:58.024408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.026221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.042291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3968d-f8ec-4fe8-9152-06482cae1063 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.071064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.071064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.071064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.071064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.071525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.071525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071b5a-fa5b-472a-8a2c-58e1b9eae8fc admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.089148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 849/1708] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:58.091154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511bbb-b51a-4e1c-82f0-d16639b5ae5a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.094218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511bbb-b51a-4e1c-82f0-d16639b5ae5a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.094649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511bbb-b51a-4e1c-82f0-d16639b5ae5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.156867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e8511bbb-b51a-4e1c-82f0-d16639b5ae5a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:58.160860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e8511bbb-b51a-4e1c-82f0-d16639b5ae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.192992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 850/1709] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:58.197208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dec5d0-b893-40dc-8d49-3c339e44fd09 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.201627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dec5d0-b893-40dc-8d49-3c339e44fd09 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.202102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dec5d0-b893-40dc-8d49-3c339e44fd09 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.203904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 851/1710] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jul 20 09:35:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:35:58.206486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dec5d0-b893-40dc-8d49-3c339e44fd09 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.217488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 860/1711] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/projects => generated 409 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:58.218343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.218642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.218924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.264309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:58.341595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:58.343267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.402979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dec5d0-b893-40dc-8d49-3c339e44fd09 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.413396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.414159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.414509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.414853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.415688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:35:58.416065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.417980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3769c8-7081-4ed1-b5e9-3e08b9fb9482 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.432959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 861/1712] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:58.439425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.452613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 852/1713] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:35:58.454316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.454610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.454938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.456653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 853/1714] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jul 20 09:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:35:58.457426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.461261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.481784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc7810-5327-4c8c-9405-4661c6127d3b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.500112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e24e-895e-4095-962a-4b83a5d06d47 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.500306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.500306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.500306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.500306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.500719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.500719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.500719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.500719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] PATH_INFO: `/v3/users/58103a8013204ddf95b468a3ae69ef9f` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.500992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.501076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58103a8013204ddf95b468a3ae69ef9f)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.502435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.505198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.557970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 862/1715] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/users => generated 311 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:58.560903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.585903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc9f4c-285c-4161-a593-fb613bdac7a9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.597004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58103a8013204ddf95b468a3ae69ef9f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:58.607633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58103a8013204ddf95b468a3ae69ef9f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:58.608965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.609798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.610083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.610364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.612159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.614183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.648419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58103a8013204ddf95b468a3ae69ef9f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:58.650276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58103a8013204ddf95b468a3ae69ef9f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:58.661592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-307fb221-bc70-4b8b-993d-4734a1c80c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58103a8013204ddf95b468a3ae69ef9f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:35:58.671699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 854/1716] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:35:58 2022] DELETE /identity/v3/users/58103a8013204ddf95b468a3ae69ef9f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:58.673401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.673932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.674825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.697237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 863/1717] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:58.701132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.704888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.722911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:58.724165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.846071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 855/1718] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:58.850052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:58.852433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.889347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:58.912464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc79054-8db9-401a-8c16-d99d571bd804 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:58.923587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:58.924593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.925060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.925682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.927938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:58.942320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:58.944730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af1348-05b9-459c-a53f-ed307ac0dcbf service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:58.974093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 864/1719] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:58.975905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:58.976298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:58.976701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:58.983908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:35:59.001413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8f59f-69bc-480f-92ac-52d6bb2f46b0 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.007123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.007850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.008128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.008400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.009421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.011023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.016032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:59.018357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.028692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 856/1720] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:59.029426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 865/1721] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:35:59.029966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.030841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.033276 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.033680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.086974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70915583-3ba6-4b1d-b60e-644def046a0c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.103551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.103879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.104172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.104336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.105013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.106934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.186245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 857/1722] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:35:59 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:35:59.188684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.210921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11950eb3-1bbe-41d2-a15d-97d0ccd32c59 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.222582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.223559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.223838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.224707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] PATH_INFO: `/v3/projects/66cdb94f6eea4def8c531d83f4c7c454/users/80cd0210d7594fc18a7b08f0b8c1f386/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.235456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fd8b4-bbac-4380-a209-4c4f8ecd29b6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.248199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.249936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cdb94f6eea4def8c531d83f4c7c454, user_id=80cd0210d7594fc18a7b08f0b8c1f386, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.251090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.252668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.253397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.254341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.274138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 858/1723] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:59 2022] PUT /identity/v3/projects/66cdb94f6eea4def8c531d83f4c7c454/users/80cd0210d7594fc18a7b08f0b8c1f386/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:59.276239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.278723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.281109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.283022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.324636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 866/1724] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:59.325890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.359000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae2112-afee-42bd-9e18-feebefaedc1a service glance] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.479976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f9b5-7890-4f6b-84b6-8b30bd7da5c6 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.487221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.487996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.488290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.489165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.489502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.491525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.494856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.495567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.496188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.496475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.522468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 867/1725] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:59.523337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.524821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.529844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.552342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 859/1726] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:35:59.553294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.581432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1c421-b9d8-4455-b4a5-c823e5100e23 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.587631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884e649-e8d6-4d6f-a9a8-a01dff60c996 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.595024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.595760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.596045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.596321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.597171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.604154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.604852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.605133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.605405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.606222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.608238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.610243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.643728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 860/1727] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:59.648989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.652412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 868/1728] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:35:59.656796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:35:59.685301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5dc7-b56e-4205-9a14-719275da52c2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.690559 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819965-0c24-454d-942f-f35fb9d73d4b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:35:59.705823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.708382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.708382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.708382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] PATH_INFO: `/v3/projects/b8f8cdc280f34f64a14b813e91bc11b7/users/511e410ce9404196aad56a7cad98855b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.711060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:35:59.712006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.712282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.712544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] PATH_INFO: `/v3/projects/66cdb94f6eea4def8c531d83f4c7c454/users/80cd0210d7594fc18a7b08f0b8c1f386/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.722363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cdb94f6eea4def8c531d83f4c7c454, user_id=80cd0210d7594fc18a7b08f0b8c1f386, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.726852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c343612-497e-4def-82f0-bd9c0ff3b3f1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.778937 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f8cdc280f34f64a14b813e91bc11b7, user_id=511e410ce9404196aad56a7cad98855b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:35:59.792328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 869/1729] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:59 2022] PUT /identity/v3/projects/66cdb94f6eea4def8c531d83f4c7c454/users/80cd0210d7594fc18a7b08f0b8c1f386/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:59.792794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1345a89-b4d2-4d02-9327-0778a94ba544 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:35:59.806610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.806610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.806610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.841599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 861/1730] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:35:59 2022] PUT /identity/v3/projects/b8f8cdc280f34f64a14b813e91bc11b7/users/511e410ce9404196aad56a7cad98855b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:35:59.853184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:35:59.853465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:35:59.853739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:35:59.891749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] MFA Rules not processed for user `80cd0210d7594fc18a7b08f0b8c1f386`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:59.893730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.943134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] MFA Rules not processed for user `511e410ce9404196aad56a7cad98855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:35:59.945721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:35:59.998300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:00.063327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 862/1731] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 20 09:36:00.067155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.103664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:00.133481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 870/1732] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:35:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:00.135756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.137581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:00.256143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894373db-54dd-40af-9736-6752b7bfe933 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.262709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.263572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.263862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.264142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.264850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.267105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.288269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56f76e-fce0-411b-ad96-558014c3a56f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.298640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.302344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.302798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.303922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.307468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 863/1733] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:00 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:00.310932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.331037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.332913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.371804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 871/1734] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:00.399000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f67cae-7388-43aa-b657-a87c47059957 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.404723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.405489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.405856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.406386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.407436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.409988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5889741-0ecd-46fe-88d7-cedbbfdde9e8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.440948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 864/1735] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:00 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:00.445232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.481311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f095638-c0ba-4116-86f3-f2a6b347443e service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.491638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.494209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.494803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.495311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.498230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.500354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.548941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 872/1736] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:00.550983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc580-626e-40fb-92f2-877b1d838658 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.551485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc580-626e-40fb-92f2-877b1d838658 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.551905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fc580-626e-40fb-92f2-877b1d838658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.560575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.611054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ef593-3c81-4313-8e6e-eedc23db8205 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.640890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.641811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.642091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.642388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] PATH_INFO: `/v3/projects/3bbe532467e5412ba80fb11b2d8b820f/users/5a0645168d834c0aa0f2ab1c06af8d65/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.658501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8a6fc580-626e-40fb-92f2-877b1d838658 None None] MFA Rules not processed for user `bc74ebac003a4f828fdb44db6dc42887`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:00.659789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8a6fc580-626e-40fb-92f2-877b1d838658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:00.700901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbe532467e5412ba80fb11b2d8b820f, user_id=5a0645168d834c0aa0f2ab1c06af8d65, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.703639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871705e0-1f88-436b-beab-d8744adbda1b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.751750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:00.756904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 873/1737] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:00 2022] PUT /identity/v3/projects/3bbe532467e5412ba80fb11b2d8b820f/users/5a0645168d834c0aa0f2ab1c06af8d65/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:00.759086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24184-3977-4512-bf22-83a9c2a9736f None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.759712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24184-3977-4512-bf22-83a9c2a9736f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.760358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a24184-3977-4512-bf22-83a9c2a9736f None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.773033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 874/1738] 10.222.0.18 () {54 vars in 912 bytes} [Wed Jul 20 09:36:00 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:36:00.775319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24184-3977-4512-bf22-83a9c2a9736f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.852719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 865/1739] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:00.854373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.854671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.854947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.900764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:00.902843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:00.951816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a24184-3977-4512-bf22-83a9c2a9736f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:00.957383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:00.958233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.958668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.958943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.960226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:00.962003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23645855-204c-4a09-ab2f-208334b47504 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:00.976870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 875/1740] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:00.978463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:00.978826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:00.979171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:00.980859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 876/1741] 10.222.0.18 () {54 vars in 961 bytes} [Wed Jul 20 09:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:36:00.981712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:00.984748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.006797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c2748-2f7f-4ae9-8cc4-84a9d0536848 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:01.012808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:01.013969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 866/1742] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:36:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:01.014084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.014555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.015262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] PATH_INFO: `/v3/projects/831c3dfee4de496981ccaf8cb0f28c7e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.017271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:01.019138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=831c3dfee4de496981ccaf8cb0f28c7e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:01.021818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:01.089750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831c3dfee4de496981ccaf8cb0f28c7e', 'request_id': 'req-5c5b599c-06da-4489-9ad1-f2f80aed4c61'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.094577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:01.098207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831c3dfee4de496981ccaf8cb0f28c7e', 'request_id': 'req-5c5b599c-06da-4489-9ad1-f2f80aed4c61'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.101465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef0a6b-fe4c-42b8-af87-7dbac98f1f14 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:01.113977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831c3dfee4de496981ccaf8cb0f28c7e', 'request_id': 'req-5c5b599c-06da-4489-9ad1-f2f80aed4c61'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.143734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] The token cache is being invalidate because project 831c3dfee4de496981ccaf8cb0f28c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:01.145839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.148092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5c5b599c-06da-4489-9ad1-f2f80aed4c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.156225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 877/1743] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:00 2022] DELETE /identity/v3/projects/831c3dfee4de496981ccaf8cb0f28c7e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:01.156225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.156225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.156225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.194283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] MFA Rules not processed for user `bc74ebac003a4f828fdb44db6dc42887`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:01.195841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.209778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:01.210443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.210722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.212775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.217160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.317270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:01.326640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 878/1744] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:01.326640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:01.328492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.348699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:01.350773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fd4ea-2c43-4561-a922-7db1e83394e5 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:01.373379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 867/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:01.376696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b36-8922-40f7-925a-f93822451455 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.376981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b36-8922-40f7-925a-f93822451455 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.377255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c602b36-8922-40f7-925a-f93822451455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.416266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6c602b36-8922-40f7-925a-f93822451455 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:01.417497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6c602b36-8922-40f7-925a-f93822451455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.469995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3487f9-e43e-488d-83ab-89fe3dda851c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:01.483141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:01.484920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.486095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.486805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] PATH_INFO: `/v3/projects/3bbe532467e5412ba80fb11b2d8b820f/users/5a0645168d834c0aa0f2ab1c06af8d65/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.491426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbe532467e5412ba80fb11b2d8b820f, user_id=5a0645168d834c0aa0f2ab1c06af8d65, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:01.495278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:01.532534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 879/1746] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:01 2022] PUT /identity/v3/projects/3bbe532467e5412ba80fb11b2d8b820f/users/5a0645168d834c0aa0f2ab1c06af8d65/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:01.532534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:01.532534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.562336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 868/1747] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Jul 20 09:36:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:01.571351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c602b36-8922-40f7-925a-f93822451455 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:01.575468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6c602b36-8922-40f7-925a-f93822451455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.603368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c602b36-8922-40f7-925a-f93822451455 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:01.673076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee174a3-98e6-4f56-bf85-1e5c96e93484 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:01.681070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:01.681749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.683099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.683635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.688178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.712318 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:01.715784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:01.747562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 880/1748] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:01.747901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:01.755556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:01.757051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:01.757167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:01.757379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] PATH_INFO: `/v3/users/90c7b7021982437b8095f01292d2f43a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:01.760399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:01.761287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c7b7021982437b8095f01292d2f43a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:01.774059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:01.867507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:01.943258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c7b7021982437b8095f01292d2f43a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:01.963524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c7b7021982437b8095f01292d2f43a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:02.000986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c7b7021982437b8095f01292d2f43a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:02.004297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c7b7021982437b8095f01292d2f43a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:02.022724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303d9b-4c5b-4085-8db0-3b6ce619492c service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:02.029763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb63314e-b387-4f61-8b37-53ac5cfe83b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c7b7021982437b8095f01292d2f43a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:02.035210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:02.037714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.038861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.039141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.043871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.052802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 869/1749] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:01 2022] DELETE /identity/v3/users/90c7b7021982437b8095f01292d2f43a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:02.054938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.056289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.056588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.064442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:02.113803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] MFA Rules not processed for user `5a0645168d834c0aa0f2ab1c06af8d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:02.116226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.295197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 870/1750] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:02.296264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:02.300891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.300981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:02.305866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:02.327272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 881/1751] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:02.328156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:02.330381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.425298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc6deb-2876-4647-bd7d-f6fe8e4be455 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:02.479015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04a069-163d-4771-badd-8ae579872c56 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:02.487840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:02.488938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.489280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.489560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.494250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.518156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:02.518948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.519235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.520689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.524738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.532944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:02.534897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955e52-622e-4550-915f-a14490feb8da service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:02.566203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 871/1752] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:02.567672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.570234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.570529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.622462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:02.648697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] MFA Rules not processed for user `a45d24d55fab4e9e852f2c65c771be17`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:02.652585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.700305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:02.701887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34386417-b3e9-41e9-9cf1-6af2dc99b265 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:02.714861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 882/1753] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:02.809470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 872/1754] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:02.820663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:02.821137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:02.821224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:02.902697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] MFA Rules not processed for user `a45d24d55fab4e9e852f2c65c771be17`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:02.904096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.938106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 883/1755] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:02.968650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:02.972290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:02.989395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:03.124426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b375094-58e3-440e-8f59-cc897a8378c0 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:03.146256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:03.149079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:03.149369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:03.152539 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] PATH_INFO: `/v3/projects/910cb01c470a4af580f7d7e601f200bd` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:03.155955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=910cb01c470a4af580f7d7e601f200bd)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:03.163726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:03.362473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:03.365543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:03.406805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910cb01c470a4af580f7d7e601f200bd', 'request_id': 'req-95fb0675-c5e2-4c4d-8050-b6f68853d602'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:03.430642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910cb01c470a4af580f7d7e601f200bd', 'request_id': 'req-95fb0675-c5e2-4c4d-8050-b6f68853d602'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:03.465896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910cb01c470a4af580f7d7e601f200bd', 'request_id': 'req-95fb0675-c5e2-4c4d-8050-b6f68853d602'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:03.527326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] The token cache is being invalidate because project 910cb01c470a4af580f7d7e601f200bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:03.527326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:03.530856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-95fb0675-c5e2-4c4d-8050-b6f68853d602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:03.535458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 873/1756] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:02 2022] DELETE /identity/v3/projects/910cb01c470a4af580f7d7e601f200bd => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:03.541124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2116-cd32-4c14-b7f7-32b6ca34fd09 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:03.541622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2116-cd32-4c14-b7f7-32b6ca34fd09 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:03.542349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a2116-cd32-4c14-b7f7-32b6ca34fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:03.603210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2d5a2116-cd32-4c14-b7f7-32b6ca34fd09 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:03.605123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2d5a2116-cd32-4c14-b7f7-32b6ca34fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:03.696729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e969-1712-4c80-b675-02ad0bb7fd02 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:03.899146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 874/1757] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:03.908812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:03.909278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:03.909716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:03.975867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:03.976558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:03.976846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:03.977240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:03.980945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:03.981777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:03.985310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:04.010464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 875/1758] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:04.012003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.033840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7e53b-271f-4767-9d63-069dae5f44cc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.082997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.086010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.086288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.086569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.087618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:04.087618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.097740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.152517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 876/1759] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:36:04.155499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.178119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.185504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.195746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:04.198613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401651c8-0052-43ec-92ac-e992f5f7b51b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.209300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 884/1760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:04.234454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.236404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.236703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.236981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.238394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.258551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4109eefa-6f94-42ab-81c6-4e7516c00520 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.352940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:04.386270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 877/1761] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:04 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:04.389250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.449482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.449569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.449865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.452519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031b09e-3461-4092-a35f-b73e875a6aba service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.506328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] MFA Rules not processed for user `bc74ebac003a4f828fdb44db6dc42887`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:04.513015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:04.660919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.665260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.665744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.666324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.668439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.673878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.687465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 878/1762] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 20 09:36:04.738132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 885/1763] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:04 2022] POST /identity/v3/users => generated 292 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:04.741685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.758469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2f0a6-9f04-45ec-ae5b-177678fa1a7e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.769802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.771046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.772115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.772562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.774614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.777328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.801151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 879/1764] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:04.804620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.815026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401993dc-8e87-44a5-8f15-b10f840fb88e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.822181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.822961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.823247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.823556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.824382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.826085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12634ce8-ee8a-45f5-9b76-b511e9d60917 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:04.840562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 886/1765] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:04.846330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:04.900975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fac88a-f965-489e-8da2-60a12069252e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:04.920139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:04.923722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:04.923984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:04.924391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] PATH_INFO: `/v3/projects/47d7c3eaea1f436a896b1a89f990a10e/users/873090328af04015824c214f505364d2/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:04.971241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d7c3eaea1f436a896b1a89f990a10e, user_id=873090328af04015824c214f505364d2, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:04.981358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:05.032034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 880/1766] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:04 2022] PUT /identity/v3/projects/47d7c3eaea1f436a896b1a89f990a10e/users/873090328af04015824c214f505364d2/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:05.034535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:05.035746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:05.036542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:05.069891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] MFA Rules not processed for user `873090328af04015824c214f505364d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:05.072013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:05.265775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 887/1767] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:05.267501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:05.437137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28865c6b-9e30-4692-b5f1-b4a5c27e577a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:05.453177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:05.454089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:05.457501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:05.458002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:05.465285 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:05.469023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:05.534722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 881/1768] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:05 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:05.539044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:05.555725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0fa9d-0852-4314-8745-4e54a105c00d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:05.733356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:05.734064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:05.734353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:05.740149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:05.741099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:05.743105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:05.768002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:05.780667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 888/1769] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:05 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:05.789200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:05.801757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd42311-1a99-4bb6-aa1c-52ab3591247f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:05.815469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:05.816179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:05.816466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:05.816747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:05.817490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:05.819252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:05.848153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 882/1770] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:05.852845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:05.873136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22197d9d-68fa-465a-b1ca-bff98ba4348d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:05.888836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:05.889393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:05.889681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:05.889964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] PATH_INFO: `/v3/projects/0c433c82f5f3403ab1541c14a17a354c/users/b8fa8ebae14c4d95b9f2351c4c3b9640/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:05.933011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c433c82f5f3403ab1541c14a17a354c, user_id=b8fa8ebae14c4d95b9f2351c4c3b9640, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:05.944717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:06.009156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 889/1771] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:05 2022] PUT /identity/v3/projects/0c433c82f5f3403ab1541c14a17a354c/users/b8fa8ebae14c4d95b9f2351c4c3b9640/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:06.013372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:06.028591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0006a8-a15b-4ee9-a703-9c21d21f270d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:06.141631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:06.190321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:06.191022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:06.191430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:06.191716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:06.192537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:06.194864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90918dd9-f51a-40ba-9867-96de1dc3403e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:06.207474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 883/1772] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:06.211740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:06.251487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc39fad-ca6e-4e87-b649-ab8a82cadc5e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:06.285236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:06.286172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:06.286433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:06.286794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] PATH_INFO: `/v3/projects/0c433c82f5f3403ab1541c14a17a354c/users/b8fa8ebae14c4d95b9f2351c4c3b9640/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:06.290838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c433c82f5f3403ab1541c14a17a354c, user_id=b8fa8ebae14c4d95b9f2351c4c3b9640, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:06.294225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:06.309818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 890/1773] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:06 2022] PUT /identity/v3/projects/0c433c82f5f3403ab1541c14a17a354c/users/b8fa8ebae14c4d95b9f2351c4c3b9640/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:06.336108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:06.336662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:06.337108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:06.448740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] MFA Rules not processed for user `b8fa8ebae14c4d95b9f2351c4c3b9640`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:06.456651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:06.726728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 884/1774] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:06.735923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:06.737838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:06.930747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89695d03-d723-447f-b8ba-99d13c5c0607 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:06.946301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:06.947480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:06.947841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:06.948194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:06.964280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:06.968076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:06.982340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 891/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:07.133891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:07.145538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c44cd-0ad7-47d6-82ab-4faaca236bde None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:07.150186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:07.150852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:07.151130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:07.151962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:07.250794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:07.252434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:07.263756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 885/1776] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:09.622420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:09.624942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:09.650309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02f9ad-349d-488b-8dc7-1d48fa5a7047 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:09.668634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:09.670316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:09.671169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:09.672053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:09.679090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:09.781958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:09.783528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07736564-61a5-47e8-b057-0b68ba7bc13d service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:09.795521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 892/1777] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:11.329312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:11.330753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:11.414198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a0189-a9ab-47c9-8315-84ba7b7d2f61 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:11.421398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:11.422647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:11.423162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:11.423988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] PATH_INFO: `/v3/users/9299822ff7074363bfa5740c1e5e6255` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:11.425021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9299822ff7074363bfa5740c1e5e6255)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:11.426961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:11.429289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:11.500809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9299822ff7074363bfa5740c1e5e6255'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:11.515422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9299822ff7074363bfa5740c1e5e6255'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:11.532639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9299822ff7074363bfa5740c1e5e6255'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:11.533376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9299822ff7074363bfa5740c1e5e6255'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:11.541969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9299822ff7074363bfa5740c1e5e6255'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:11.548411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 886/1778] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:11 2022] DELETE /identity/v3/users/9299822ff7074363bfa5740c1e5e6255 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:11.553100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:11.553601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:11.553695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:11.575482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:11.577163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:11.642738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:11.657291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 893/1779] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:11.667423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:11.668851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:11.741156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c05c1-d064-4539-a419-86e27b402c12 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:11.746824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:11.747546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:11.747835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:11.748250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:11.761467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:11.763353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:11.778023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 887/1780] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:12.165809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:12.167657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:12.191347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee6a85-bffb-4785-ae1d-5b067f966569 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:12.196852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:12.197807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:12.198133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:12.198494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] PATH_INFO: `/v3/projects/4479dc9a15cc422fa0d4120ec03d9785` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:12.199716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4479dc9a15cc422fa0d4120ec03d9785)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:12.202053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:12.262884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4479dc9a15cc422fa0d4120ec03d9785', 'request_id': 'req-ae2e4487-50dd-4255-ad61-18589b947167'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.268700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4479dc9a15cc422fa0d4120ec03d9785', 'request_id': 'req-ae2e4487-50dd-4255-ad61-18589b947167'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.276528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4479dc9a15cc422fa0d4120ec03d9785', 'request_id': 'req-ae2e4487-50dd-4255-ad61-18589b947167'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.291061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] The token cache is being invalidate because project 4479dc9a15cc422fa0d4120ec03d9785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:12.291529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.292324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.294208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 894/1781] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:12 2022] DELETE /identity/v3/projects/4479dc9a15cc422fa0d4120ec03d9785 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:12.299609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:12.311539 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e385-3dfe-4da1-94b5-a191d737dd96 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:12.381933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:12.382850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:12.383446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:12.383726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] PATH_INFO: `/v3/users/80cd0210d7594fc18a7b08f0b8c1f386` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:12.384944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80cd0210d7594fc18a7b08f0b8c1f386)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:12.386899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:12.416897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:12.479198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80cd0210d7594fc18a7b08f0b8c1f386'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.500880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80cd0210d7594fc18a7b08f0b8c1f386'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.514948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80cd0210d7594fc18a7b08f0b8c1f386'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.515810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80cd0210d7594fc18a7b08f0b8c1f386'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.525057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-883a37e5-ee16-41ce-b802-ed8e7714b155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80cd0210d7594fc18a7b08f0b8c1f386'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:12.532680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 888/1782] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:12 2022] DELETE /identity/v3/users/80cd0210d7594fc18a7b08f0b8c1f386 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:12.872035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:12.873560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:12.886960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4487-50dd-4255-ad61-18589b947167 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:12.948198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:12.949086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:12.949377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:12.949686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] PATH_INFO: `/v3/projects/66cdb94f6eea4def8c531d83f4c7c454` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:12.951050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cdb94f6eea4def8c531d83f4c7c454)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:12.953343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.006072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cdb94f6eea4def8c531d83f4c7c454', 'request_id': 'req-d9015c02-d010-4b85-be04-84c87fe94d24'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:13.011648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cdb94f6eea4def8c531d83f4c7c454', 'request_id': 'req-d9015c02-d010-4b85-be04-84c87fe94d24'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:13.019211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cdb94f6eea4def8c531d83f4c7c454', 'request_id': 'req-d9015c02-d010-4b85-be04-84c87fe94d24'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:13.030798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] The token cache is being invalidate because project 66cdb94f6eea4def8c531d83f4c7c454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:13.031083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:13.031801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d9015c02-d010-4b85-be04-84c87fe94d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:13.033428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 895/1783] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:12 2022] DELETE /identity/v3/projects/66cdb94f6eea4def8c531d83f4c7c454 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:13.039279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.039602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.039909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.067968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:13.069126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:13.132644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 889/1784] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:13.141365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.141655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.141933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.160943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:13.162625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:13.170522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 896/1785] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:13.176406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.187220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2804dcf-a66c-4b71-b34d-bf915231136a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.191813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.192474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.193386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.193667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.194390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:13.194695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.196348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.208515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 890/1786] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:13.212943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.218957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:13.223493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c6c85-1942-40ff-a484-540ba27bfcd2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.230924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.231703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.232055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.232397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.233231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.235250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.263659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 897/1787] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:13.268584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.279542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071729-703b-4883-a32e-ba9cfe4e8345 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.340887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.341564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.341845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.342114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.342799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.344740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.369641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 891/1788] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:13.370804 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.381205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c19b0-1325-4c7c-b2f0-df642ef97e61 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.386281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.386958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.387239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.387546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.388271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.389888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.402626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 898/1789] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:13.406923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.418922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca922fc2-0a0a-4e70-bdf0-dbe4aaa0f180 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.423528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.424187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.424467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.424738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.425452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.427108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf1437-94fa-430c-86fb-3daada6a02b5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.438558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 892/1790] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:13.442782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.460992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5443c63-0f8b-4671-bcdf-d015f453717a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.469787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.470592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.470874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.471143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] PATH_INFO: `/v3/projects/7a1c6df9d99c45ad8f6a98d9385a075f/users/f3621ea56eb0418dbaea5b71ba4ca1d9/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.491048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1c6df9d99c45ad8f6a98d9385a075f, user_id=f3621ea56eb0418dbaea5b71ba4ca1d9, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.493894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.510970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 899/1791] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:13 2022] PUT /identity/v3/projects/7a1c6df9d99c45ad8f6a98d9385a075f/users/f3621ea56eb0418dbaea5b71ba4ca1d9/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:13.522657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da6a9-12ce-4840-9875-17c72dc6c9ec None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.522943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da6a9-12ce-4840-9875-17c72dc6c9ec None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.523219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da6a9-12ce-4840-9875-17c72dc6c9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.557783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:13.599982 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-448da6a9-12ce-4840-9875-17c72dc6c9ec None None] MFA Rules not processed for user `f3621ea56eb0418dbaea5b71ba4ca1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:13.602459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-448da6a9-12ce-4840-9875-17c72dc6c9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:13.765080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 893/1792] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:13.770273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:13.903926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a24c2-9981-4e81-8473-96e1a9264586 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:13.913897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:13.914843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.915491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.915983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.917084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:13.921507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:13.941996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:13.942347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:13.942771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:13.954798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 900/1793] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:13.959706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.014160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:14.015509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:14.089845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ce349-e26e-46e3-a2af-c9e6e4a2913d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.121898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.122607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.122890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.131461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.134452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.138858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.144489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 894/1794] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:14.183041 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 901/1795] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:13 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:14.187123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.222626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ed39c-fbf0-4f58-8268-16d79b5c3fed None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.254408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.256136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.256511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.257665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.260488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.260911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.262848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:14.263821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.279323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 895/1796] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:14.284506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.298508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8632fda-ed9b-40fa-87a3-b8ca58bbcdb3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.307205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.307971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.308257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.308698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.310076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.311375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.312661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:14.342300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 896/1797] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:14.345586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.357174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603756e1-8816-4172-a48f-4aea4fddfc66 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.369583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.370477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.370764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.371071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] PATH_INFO: `/v3/projects/2d252ff3a92d485b9c01700bcc58d021/users/4175fa8e1bf44d4f8bae5f2585096c10/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.393839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d252ff3a92d485b9c01700bcc58d021, user_id=4175fa8e1bf44d4f8bae5f2585096c10, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.397143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb3320-a205-47fb-a6b1-09360025600c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.415769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 897/1798] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:14 2022] PUT /identity/v3/projects/2d252ff3a92d485b9c01700bcc58d021/users/4175fa8e1bf44d4f8bae5f2585096c10/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:14.427319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.427612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.427940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.464405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] MFA Rules not processed for user `4175fa8e1bf44d4f8bae5f2585096c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:14.465898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:14.469101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ad489-98a2-4071-b108-f2ccc82bfe79 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.481516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:14.560643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.561319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.561604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.561881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.567658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 898/1799] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:14.569117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.645144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bc117-c2a0-4577-8310-e9e4345c22fd None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.650988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.651714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.652238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.652238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.652993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.655011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.698308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 899/1800] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:14 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:14.699779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.704798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.706447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.748872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 902/1801] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:36:14.822601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5a867-5e65-43db-b6fa-97eb6817c913 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.828495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.829370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.829665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.829992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.830753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.833822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.868314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 900/1802] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:14 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:14.869295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.894408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195599b-ab25-4b52-a7b3-12e64e94291a service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.905585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.906650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.906935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.908878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:14.909712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:14.914924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:14.952286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:14.962627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 903/1803] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:14.964296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b5ca0-2958-4cc6-9cdb-e35146644e67 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:14.971791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:14.974629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:14.975084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:14.975567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] PATH_INFO: `/v3/projects/ab1c6963496e418d944851022c64d904/users/ec10798282394ca680eca1a449d18700/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.004152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1c6963496e418d944851022c64d904, user_id=ec10798282394ca680eca1a449d18700, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:15.007934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:15.034510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 901/1804] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:14 2022] PUT /identity/v3/projects/ab1c6963496e418d944851022c64d904/users/ec10798282394ca680eca1a449d18700/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:15.039217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:15.052493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f705488-4b85-4be3-83ac-8f7d5f8351fd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:15.096112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:15.126008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:15.126713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:15.127123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:15.127589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.128379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:15.130732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443f30d-ca84-4314-92fe-1d728ad01661 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:15.159403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 904/1805] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:15.163983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:15.169360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:15.174929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6c185-4a98-43c9-8862-74bcffa36fa5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:15.181118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:15.182018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:15.182300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:15.182576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] PATH_INFO: `/v3/projects/ab1c6963496e418d944851022c64d904/users/ec10798282394ca680eca1a449d18700/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.184724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1c6963496e418d944851022c64d904, user_id=ec10798282394ca680eca1a449d18700, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:15.187485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:15.198831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 902/1806] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:15 2022] PUT /identity/v3/projects/ab1c6963496e418d944851022c64d904/users/ec10798282394ca680eca1a449d18700/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:15.211231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:15.211606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:15.211888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.245773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] MFA Rules not processed for user `ec10798282394ca680eca1a449d18700`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:15.247491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:15.335888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 905/1807] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:15.348474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:15.351159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:15.419178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648fa8c-1caf-4322-89c9-3279540744f9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:15.423169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:15.424077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:15.424461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:15.424830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.435528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:15.437208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:15.450093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 903/1808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:15.493220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:15.495871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:15.507997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab23bd-bd5a-41dd-91a7-487dc363ee32 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:15.516140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:15.516766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:15.517101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:15.517379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:15.576308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:15.577738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561f9fa-7911-47c1-8b89-af6703dea4af service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:15.586897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 906/1809] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:17.382932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:17.384355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:17.470510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa75376-bb4b-41f2-a351-b7ce24599377 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:17.476665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:17.477688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:17.478323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:17.478879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] PATH_INFO: `/v3/users/873090328af04015824c214f505364d2` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:17.480723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=873090328af04015824c214f505364d2)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:17.484233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:17.555223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873090328af04015824c214f505364d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:17.564215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873090328af04015824c214f505364d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:17.577196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873090328af04015824c214f505364d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:17.577890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873090328af04015824c214f505364d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:17.586529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873090328af04015824c214f505364d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:17.598083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 904/1810] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:17 2022] DELETE /identity/v3/users/873090328af04015824c214f505364d2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:17.600472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:17.600997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:17.601204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:17.620689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:17.621915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:17.754265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:17.783556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 907/1811] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:17.795361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:17.796740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:17.810097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:17.856675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72af14-c22c-43d6-8dfa-00418ed29603 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:17.860223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:17.860863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:17.861141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:17.861408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:17.870663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:17.872122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:17.882333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 905/1812] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:18.178690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:18.180298 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:18.192861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9abe4-c6a6-4abf-ad02-e5a1466d5192 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:18.199343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:18.200506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:18.200869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:18.201219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] PATH_INFO: `/v3/projects/47d7c3eaea1f436a896b1a89f990a10e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:18.203138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d7c3eaea1f436a896b1a89f990a10e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:18.205146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:18.277956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d7c3eaea1f436a896b1a89f990a10e', 'request_id': 'req-1d299b17-a70b-4df0-a03f-615f3c7e3d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.290172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d7c3eaea1f436a896b1a89f990a10e', 'request_id': 'req-1d299b17-a70b-4df0-a03f-615f3c7e3d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.300887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d7c3eaea1f436a896b1a89f990a10e', 'request_id': 'req-1d299b17-a70b-4df0-a03f-615f3c7e3d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.315497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] The token cache is being invalidate because project 47d7c3eaea1f436a896b1a89f990a10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:18.315906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.317031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.324625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 908/1813] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:18 2022] DELETE /identity/v3/projects/47d7c3eaea1f436a896b1a89f990a10e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:18.326348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:18.345204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43872313-d34e-491c-a1db-a0689b2d7e0f service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:18.499672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:18.500796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:18.501171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:18.501540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] PATH_INFO: `/v3/users/b8fa8ebae14c4d95b9f2351c4c3b9640` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:18.502877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8fa8ebae14c4d95b9f2351c4c3b9640)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:18.504984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:18.564469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8fa8ebae14c4d95b9f2351c4c3b9640'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.574092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8fa8ebae14c4d95b9f2351c4c3b9640'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.587754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8fa8ebae14c4d95b9f2351c4c3b9640'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.588492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8fa8ebae14c4d95b9f2351c4c3b9640'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.600939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7ec1a4d0-fed9-4e66-95d3-b57fe98642e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8fa8ebae14c4d95b9f2351c4c3b9640'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:18.603114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 906/1814] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:18 2022] DELETE /identity/v3/users/b8fa8ebae14c4d95b9f2351c4c3b9640 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:18.928203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:18.930781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:18.953920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d299b17-a70b-4df0-a03f-615f3c7e3d65 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.023784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.024695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.024981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.025254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] PATH_INFO: `/v3/projects/0c433c82f5f3403ab1541c14a17a354c` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.026523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c433c82f5f3403ab1541c14a17a354c)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.028481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.098767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c433c82f5f3403ab1541c14a17a354c', 'request_id': 'req-0c852de9-974d-4729-8350-43f598c277f2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:19.105539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c433c82f5f3403ab1541c14a17a354c', 'request_id': 'req-0c852de9-974d-4729-8350-43f598c277f2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:19.116020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c433c82f5f3403ab1541c14a17a354c', 'request_id': 'req-0c852de9-974d-4729-8350-43f598c277f2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:19.130471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] The token cache is being invalidate because project 0c433c82f5f3403ab1541c14a17a354c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:19.131142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:19.132022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c852de9-974d-4729-8350-43f598c277f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:19.133766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 909/1815] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:18 2022] DELETE /identity/v3/projects/0c433c82f5f3403ab1541c14a17a354c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:19.146693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.146979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.147254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.167921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:19.169130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:19.200863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:19.235057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 907/1816] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:19.244377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.244668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.244997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.265834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:19.267197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:19.276684 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 910/1817] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:19.280878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.291643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096b338-6ba9-4cd7-9124-64c843cf8025 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.296281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.296956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.297242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.297512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.298236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:19.298542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.300188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.313034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 908/1818] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:36:19.317247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.321976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:19.329618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889da4b-b09e-4f2e-bc66-faf75dbf9b9a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.337235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.338077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.338580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.339123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.339987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.341913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.370516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 911/1819] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:19.374639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.385490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df734db-9738-40ec-bdbb-c0aaa4f6024b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.460084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.460233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.460500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.461124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.461449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.464647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.517390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 909/1820] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:19.526201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.543040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c8274-80fb-49c8-8461-af2f0bc5ca84 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.548738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.549439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.549716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.549992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.550734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.553841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.570606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 912/1821] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:19.574257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.586698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6f0e6-e10d-4686-9776-a71735ef25e4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.594206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.595554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.596158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.596722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.598128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.600805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37621e-b0f3-41fc-b111-9485295c56f3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.619059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 910/1822] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:19.624313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.638964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad385ee2-d1fe-4d69-bb0e-9507263958c8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.652467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.653623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.653975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.654320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] PATH_INFO: `/v3/projects/bf5e4976e9e14c7b8c91241c5cf05f00/users/e770febd620a4a689c473ebf0f05509b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.676493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5e4976e9e14c7b8c91241c5cf05f00, user_id=e770febd620a4a689c473ebf0f05509b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.680424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.698919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 913/1823] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:19 2022] PUT /identity/v3/projects/bf5e4976e9e14c7b8c91241c5cf05f00/users/e770febd620a4a689c473ebf0f05509b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:19.709942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.710455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.710776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.754141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] MFA Rules not processed for user `e770febd620a4a689c473ebf0f05509b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:19.756383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:19.841719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 911/1824] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:19.846091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.919200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54be0bb-223c-42d4-93f5-b1dabb51340c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:19.924093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:19.924796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:19.925078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:19.925379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:19.926112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:19.928674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:19.962950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:19.965814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 914/1825] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:19.975219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95225223-70d9-47b3-9ee2-495172820b05 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.002888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:20.050856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.051680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.051964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.052357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.053056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.054995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.076056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 912/1826] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:19 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:20.080304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.096212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f34da-4fd7-4350-9617-2a0979268d03 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.102297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.102831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.103122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.103591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.104230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.105914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.120479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 915/1827] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:20.126966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.139747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4b3b-83a0-4153-aa81-cd4a6be3b211 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.151691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.152172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.152857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.153209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] PATH_INFO: `/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663/users/f058df2213184c8eaf79007ccf61832c/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.177064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bbaef6a65a4411aa0e136cfc4b6663, user_id=f058df2213184c8eaf79007ccf61832c, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.180897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.199330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 913/1828] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:20 2022] PUT /identity/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663/users/f058df2213184c8eaf79007ccf61832c/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:20.202700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.215960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fdcc68-01f5-435b-86b4-b29558399019 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.286235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.287871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.288308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.288582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.289304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.290887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79290f-7f4e-4385-99e3-07735f07c987 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.305564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 916/1829] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:20.307273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.318839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a0ae6-d629-4785-969b-16e7d9e71835 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.323860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.325140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.325423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.325704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] PATH_INFO: `/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663/users/f058df2213184c8eaf79007ccf61832c/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.328023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bbaef6a65a4411aa0e136cfc4b6663, user_id=f058df2213184c8eaf79007ccf61832c, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.330704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.342449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 914/1830] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:20 2022] PUT /identity/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663/users/f058df2213184c8eaf79007ccf61832c/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:20.354277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.354536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.354818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.382794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] MFA Rules not processed for user `f058df2213184c8eaf79007ccf61832c`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:20.384046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:20.388341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:20.479760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 917/1831] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:20.488286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.490804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:20.550465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f258bd-3004-4c7a-a794-1cd6b30c03a0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.554377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.555045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.555355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.555658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.576793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.578273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd90f2-816c-418c-b75e-5a35085b315a service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:20.588610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 915/1832] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:20.748368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:20.889982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da98fbe-65ba-4f15-9d25-be3402974594 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:20.909659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:20.910572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:20.911005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:20.911329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] PATH_INFO: `/v3/users/e770febd620a4a689c473ebf0f05509b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:20.916240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e770febd620a4a689c473ebf0f05509b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:20.918248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:21.050021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e770febd620a4a689c473ebf0f05509b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.064822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e770febd620a4a689c473ebf0f05509b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.065471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e770febd620a4a689c473ebf0f05509b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.088806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e770febd620a4a689c473ebf0f05509b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.098050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e770febd620a4a689c473ebf0f05509b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.117734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 918/1833] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:20 2022] DELETE /identity/v3/users/e770febd620a4a689c473ebf0f05509b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:21.120099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:21.120410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:21.120757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:21.145950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:21.149031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:21.293159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 916/1834] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:21.301883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:21.306900 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:21.364845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:21.441266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58edba4-3de6-40a5-bb30-bd12280b33f3 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:21.447973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:21.449114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:21.449608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:21.450099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:21.464464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:21.465901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:21.482336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 919/1835] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:21.865521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:21.867356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:21.880035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305e013-48cb-4eec-8482-d57431a65193 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:21.885905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:21.886835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:21.887670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:21.887994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] PATH_INFO: `/v3/projects/bf5e4976e9e14c7b8c91241c5cf05f00` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:21.889380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5e4976e9e14c7b8c91241c5cf05f00)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:21.891312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:21.932491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:21.973064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5e4976e9e14c7b8c91241c5cf05f00', 'request_id': 'req-ed112b9f-aa20-436b-9a08-d6a2987f09b8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.984562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5e4976e9e14c7b8c91241c5cf05f00', 'request_id': 'req-ed112b9f-aa20-436b-9a08-d6a2987f09b8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:21.992206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5e4976e9e14c7b8c91241c5cf05f00', 'request_id': 'req-ed112b9f-aa20-436b-9a08-d6a2987f09b8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.006315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] The token cache is being invalidate because project bf5e4976e9e14c7b8c91241c5cf05f00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:22.006792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.008726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.010344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 917/1836] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:21 2022] DELETE /identity/v3/projects/bf5e4976e9e14c7b8c91241c5cf05f00 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:22.014609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:22.026959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ac9ee-7a25-4041-a7a0-0f519421303f service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:22.098452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:22.099929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:22.100228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:22.100678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] PATH_INFO: `/v3/users/f058df2213184c8eaf79007ccf61832c` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:22.101898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f058df2213184c8eaf79007ccf61832c)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:22.103868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:22.174778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f058df2213184c8eaf79007ccf61832c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.184461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f058df2213184c8eaf79007ccf61832c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.185193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f058df2213184c8eaf79007ccf61832c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.216560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f058df2213184c8eaf79007ccf61832c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.224919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f90dba53-b244-426f-8a0b-1c299c5a135e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f058df2213184c8eaf79007ccf61832c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.240765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 920/1837] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:22 2022] DELETE /identity/v3/users/f058df2213184c8eaf79007ccf61832c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:22.637210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:22.638647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:22.676744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed112b9f-aa20-436b-9a08-d6a2987f09b8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:22.750225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:22.751258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:22.751579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:22.751857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] PATH_INFO: `/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:22.753291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bbaef6a65a4411aa0e136cfc4b6663)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:22.755333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:22.824270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bbaef6a65a4411aa0e136cfc4b6663', 'request_id': 'req-887944d5-4aca-4d24-84d1-2418d45661d0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.841411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bbaef6a65a4411aa0e136cfc4b6663', 'request_id': 'req-887944d5-4aca-4d24-84d1-2418d45661d0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.850388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bbaef6a65a4411aa0e136cfc4b6663', 'request_id': 'req-887944d5-4aca-4d24-84d1-2418d45661d0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.862644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] The token cache is being invalidate because project b6bbaef6a65a4411aa0e136cfc4b6663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:22.863097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.863947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-887944d5-4aca-4d24-84d1-2418d45661d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:22.868281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 918/1838] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:22 2022] DELETE /identity/v3/projects/b6bbaef6a65a4411aa0e136cfc4b6663 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:22.873067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:22.873426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:22.873775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:22.893712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:22.895205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:22.947989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:22.989104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 921/1839] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:22.998131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:22.998627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:22.998980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.024679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:23.026094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:23.039820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 919/1840] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:23.044186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.064988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bf218-374a-4fbf-b666-45c8ebb2801b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.070998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.071717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.071998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.072281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.073540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:23.073854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.075679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.087768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 922/1841] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:23.092176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.101575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b1dde-30df-4ba5-a839-d4b93af6fbf5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.106689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.107397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.107681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.107959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.108654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.110578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.147096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 920/1842] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:23 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:23.148329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.158493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d525a-b22a-46e5-a57a-68603f44185c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.214438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.215055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.215377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.215683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.216382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.218315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.246540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 923/1843] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:23 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:23.248821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.263981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca595d0-becc-4f7b-aadb-25f1fddf8cc7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.269856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.270580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.271003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.271315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.272049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.273743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.276388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:23.289680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 921/1844] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:23.295070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.323829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88078adf-bddd-4d4b-ad4e-deb79dc0e204 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.330026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.330710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.335817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.336103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.336898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.348309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ce0d5-511d-43a4-a589-5ec7bec70824 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.369965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 924/1845] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:23.371483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.382897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a4594-8092-4efb-9a08-387f063b8778 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.388212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.389412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.390195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.390195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] PATH_INFO: `/v3/projects/298d7611d92a47fd94dd66c8ff10a7f3/users/7ed56ed8be784927917986375a76df4c/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.416438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298d7611d92a47fd94dd66c8ff10a7f3, user_id=7ed56ed8be784927917986375a76df4c, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.419182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.442409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 922/1846] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:23 2022] PUT /identity/v3/projects/298d7611d92a47fd94dd66c8ff10a7f3/users/7ed56ed8be784927917986375a76df4c/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:23.448514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.448811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.449079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.484832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:23.486088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:23.574984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 925/1847] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:23.588925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:23.590647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:23.702832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a3fc9-942d-42cf-bbdf-67fb5e1a7843 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:23.719588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:23.720645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:23.721048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:23.721435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:23.735512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:23.737487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a2737-380f-4594-abf6-37307de48ca2 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:23.750139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 923/1848] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:24.107388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:24.110307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:24.133451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40223078-6a88-4a96-9c13-ce1c002d8213 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:24.140860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:24.142203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:24.142674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:24.143131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] PATH_INFO: `/v3/users/f3621ea56eb0418dbaea5b71ba4ca1d9` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:24.144686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3621ea56eb0418dbaea5b71ba4ca1d9)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:24.147195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:24.258659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:24.262173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3621ea56eb0418dbaea5b71ba4ca1d9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:24.276592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3621ea56eb0418dbaea5b71ba4ca1d9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:24.278250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3621ea56eb0418dbaea5b71ba4ca1d9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:24.307719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3621ea56eb0418dbaea5b71ba4ca1d9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:24.318689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3621ea56eb0418dbaea5b71ba4ca1d9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:24.329330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 926/1849] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:24 2022] DELETE /identity/v3/users/f3621ea56eb0418dbaea5b71ba4ca1d9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:24.332333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0797-77a9-4240-af36-cb8b64abdc26 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:24.332847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0797-77a9-4240-af36-cb8b64abdc26 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:24.333021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd0797-77a9-4240-af36-cb8b64abdc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:24.357215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b3dd0797-77a9-4240-af36-cb8b64abdc26 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:24.366309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b3dd0797-77a9-4240-af36-cb8b64abdc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:24.479972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 924/1850] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:24.491697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:24.493330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:24.580439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442814a-408f-4851-8f86-98947995ca0f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:24.588368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:24.589289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:24.589729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:24.590253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:24.609467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:24.611762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:24.656054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 927/1851] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:25.003962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.004597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.004597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.074371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] MFA Rules not processed for user `7ed56ed8be784927917986375a76df4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:25.076327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.079916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:25.081587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.104971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280498b1-482f-443a-99b6-022eaf1ef951 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:25.115014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:25.116215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.116502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.116778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] PATH_INFO: `/v3/projects/7a1c6df9d99c45ad8f6a98d9385a075f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.119217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1c6df9d99c45ad8f6a98d9385a075f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:25.121761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:25.157510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 925/1852] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:25.170187 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:25.172502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.189200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:25.214277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1c6df9d99c45ad8f6a98d9385a075f', 'request_id': 'req-e442c5df-6db5-4583-95a1-e54e706d3502'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.226199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1c6df9d99c45ad8f6a98d9385a075f', 'request_id': 'req-e442c5df-6db5-4583-95a1-e54e706d3502'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.240786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1c6df9d99c45ad8f6a98d9385a075f', 'request_id': 'req-e442c5df-6db5-4583-95a1-e54e706d3502'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.262912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] The token cache is being invalidate because project 7a1c6df9d99c45ad8f6a98d9385a075f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:25.263643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.265037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.272147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 928/1853] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:25 2022] DELETE /identity/v3/projects/7a1c6df9d99c45ad8f6a98d9385a075f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:25.273851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:25.276912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.388300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:25.392508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dd261-4dda-42d3-94f9-c53a7022ee3d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:25.416311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442c5df-6db5-4583-95a1-e54e706d3502 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:25.424001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:25.425398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.425676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.426025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] PATH_INFO: `/v3/users/4175fa8e1bf44d4f8bae5f2585096c10` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.427928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4175fa8e1bf44d4f8bae5f2585096c10)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:25.430094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:25.464868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:25.478782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.479077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.479394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.486525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.522134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4175fa8e1bf44d4f8bae5f2585096c10'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.541971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4175fa8e1bf44d4f8bae5f2585096c10'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.543542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4175fa8e1bf44d4f8bae5f2585096c10'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.574516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4175fa8e1bf44d4f8bae5f2585096c10'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.586650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:25.588184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f6616-c18e-4621-8d21-048361666914 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:25.600011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4175fa8e1bf44d4f8bae5f2585096c10'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:25.608559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 926/1854] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:25.609153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 929/1855] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:25 2022] DELETE /identity/v3/users/4175fa8e1bf44d4f8bae5f2585096c10 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:25.610518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.610811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.611088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.651848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:25.655389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.790490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 927/1856] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Jul 20 09:36:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:25.829135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:25.831385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.837439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:25.840448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:25.845212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fddef-d0c0-47f2-8b28-ce590f9189bb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:25.850351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:25.851193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.851600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.851966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.860457 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24c25b-5cc3-4d16-beb7-62300082de03 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:25.864259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:25.867440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:25.871168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:25.871966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:25.872416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:25.872853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:25.889700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:25.891558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:25.892180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 930/1857] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:25.904541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 928/1858] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:26.315046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:26.316708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:26.464648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d0895-1d64-4ea7-aa0b-433fa7445c4a service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:26.470813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:26.472248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:26.472530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:26.472982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] PATH_INFO: `/v3/projects/2d252ff3a92d485b9c01700bcc58d021` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:26.474910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d252ff3a92d485b9c01700bcc58d021)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:26.477518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:26.601005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d252ff3a92d485b9c01700bcc58d021', 'request_id': 'req-f538363e-458b-44e4-8523-9b0e70ddc538'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.606724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:26.608636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d252ff3a92d485b9c01700bcc58d021', 'request_id': 'req-f538363e-458b-44e4-8523-9b0e70ddc538'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.624799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d252ff3a92d485b9c01700bcc58d021', 'request_id': 'req-f538363e-458b-44e4-8523-9b0e70ddc538'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.649493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] The token cache is being invalidate because project 2d252ff3a92d485b9c01700bcc58d021 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:26.649928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.651690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f538363e-458b-44e4-8523-9b0e70ddc538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.659935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:26.667858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 931/1859] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:26 2022] DELETE /identity/v3/projects/2d252ff3a92d485b9c01700bcc58d021 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:26.679206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26f5ea-8936-4797-b968-a1682911adde service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:26.697332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:26.775970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:26.777068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:26.777345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:26.777794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] PATH_INFO: `/v3/users/ec10798282394ca680eca1a449d18700` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:26.779376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec10798282394ca680eca1a449d18700)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:26.782227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:26.881432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec10798282394ca680eca1a449d18700'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.895535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec10798282394ca680eca1a449d18700'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.919994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec10798282394ca680eca1a449d18700'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.920930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec10798282394ca680eca1a449d18700'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.932991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ef955336-7f48-43b2-8023-bf2de8b865e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec10798282394ca680eca1a449d18700'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:26.940979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 929/1860] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:26 2022] DELETE /identity/v3/users/ec10798282394ca680eca1a449d18700 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:27.098252 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:27.098532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:27.099048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:27.100653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 932/1861] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:36:27.113887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:27.114829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:27.116243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:27.164811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:27.166750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:27.286177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 930/1862] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:36:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:27.312073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:27.313848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:27.337313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7ffde-da0f-4793-a425-87b680260fe8 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:27.343622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:27.344328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:27.344640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:27.345026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:27.369918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:27.373567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981949cc-2549-46f1-afab-aba79caa9754 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:27.385697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 933/1863] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:27.907647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:27.908297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:28.011310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:28.112079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2255fce-65d6-4c93-93d1-60d53c3f1763 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.124081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.124943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.125257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.125668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] PATH_INFO: `/v3/projects/ab1c6963496e418d944851022c64d904` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.136353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1c6963496e418d944851022c64d904)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.138736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.216643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1c6963496e418d944851022c64d904', 'request_id': 'req-5c2c91a7-7f24-4cad-9728-b429bb8601dd'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:28.240499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1c6963496e418d944851022c64d904', 'request_id': 'req-5c2c91a7-7f24-4cad-9728-b429bb8601dd'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:28.256531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1c6963496e418d944851022c64d904', 'request_id': 'req-5c2c91a7-7f24-4cad-9728-b429bb8601dd'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:28.282532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] The token cache is being invalidate because project ab1c6963496e418d944851022c64d904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:28.282829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:28.289660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5c2c91a7-7f24-4cad-9728-b429bb8601dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:28.299759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 931/1864] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:27 2022] DELETE /identity/v3/projects/ab1c6963496e418d944851022c64d904 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:28.306455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.306754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.307095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.339619 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:28.342489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:28.421105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 934/1865] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:28.430035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.430325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.430633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.465829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:28.467651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:28.478314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 932/1866] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:28.483053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.496199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d8ffa-94fa-4f79-b525-22d35b95632a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.501586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.502942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.504104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.504726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.506232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:28.506783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.508917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.531372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.537144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 935/1867] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:28.543194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63cfff-d8bf-453e-8a76-62dfe8d2d82f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.549646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.550320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.550605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.550883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.551626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.553640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371dc8e-cff8-47cb-be82-a2721afde67b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.585755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 933/1868] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:28.589998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.602326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51218-6276-4a05-a776-9b89daddc94a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.663970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc877-bf1f-4129-a040-7007fd51e6b9 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.664732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc877-bf1f-4129-a040-7007fd51e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.665290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc877-bf1f-4129-a040-7007fd51e6b9 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.667571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 934/1869] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:36:28.683493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.688112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.688436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.722045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.722888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.723334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.723882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.724737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.726895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.728161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:28.732407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:28.768859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 936/1870] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:28.773063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.789877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31dcb-caf2-4871-ab76-3a1de8384afa admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.798563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.800150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.800446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.800730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.803845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.805488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.829263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 937/1871] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:28.832854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.846445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30f15-4553-42eb-b0b7-5a9ce574ba8b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.854148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.854835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.855464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.855932 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.856824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.857295 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 935/1872] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:28.858886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1f79e-7687-4f6a-85bf-1e2b596e7f8d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.880074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 938/1873] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:28.881335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:28.900019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dda10-3fa7-4c8b-adce-41ca42ff8534 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:28.905927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:28.907040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.907417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.907896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] PATH_INFO: `/v3/projects/275d6205355741dca52d516e800aa5c6/users/f80229361211489eb5c1dcd49d170cf3/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:28.935262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275d6205355741dca52d516e800aa5c6, user_id=f80229361211489eb5c1dcd49d170cf3, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:28.939242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:28.942554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:28.960629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 936/1874] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:28 2022] PUT /identity/v3/projects/275d6205355741dca52d516e800aa5c6/users/f80229361211489eb5c1dcd49d170cf3/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:28.973198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:28.973750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:28.974095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.007064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] MFA Rules not processed for user `f80229361211489eb5c1dcd49d170cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:29.008361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:29.024050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:29.087716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 939/1875] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:29.092312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.100647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.102664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:29.171618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361b6f9-4543-440c-a246-33f3bc0d9020 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.180382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.181302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.181748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.182160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.183520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.187551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.202779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a4a5-69a3-4683-bfa9-3a58308e6070 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.210105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.210796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.211220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.211529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.223558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.226062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.227743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 937/1876] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:29 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:29.232735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.240925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 940/1877] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:29.321102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5d089-4b2d-420a-9b7f-0705e3b86c97 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.327301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.328229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.328610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.328980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.329768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.332499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518ac43-a4cc-421c-93c6-fa50e19c185f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.362905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 938/1878] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:29 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:29.368489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.382720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ca1c-fecd-4f8a-b21b-eb01afdd5bf4 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.403009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.403736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.404021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.404138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.404324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.404594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.404961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.405072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.406681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.422498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 941/1879] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:29.423198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.430850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] MFA Rules not processed for user `7ed56ed8be784927917986375a76df4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:29.437151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398e310-ffe3-4d80-922e-17aedd466dc9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.437319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:29.449360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.450285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.451216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.451216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] PATH_INFO: `/v3/projects/05802417523e4d33b4792e28755ab498/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.475239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05802417523e4d33b4792e28755ab498, user_id=b0a4e91cd34a4c8a90c8a8ebdf58aa97, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.478040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.497546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 942/1880] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:29 2022] PUT /identity/v3/projects/05802417523e4d33b4792e28755ab498/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:29.504348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.553413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 939/1881] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:29.607801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e68ad-ef44-4431-bde2-aa1a4e4d2cf7 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.613978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.616079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.616560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.617012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.617941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.621469 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a67222-4bb0-415e-8e2e-b00b47c63f72 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.629295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:29.638878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 943/1882] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:29.642161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.654352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ff2f7-4f03-4869-8e50-8029ab4f0cb5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.659787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.660802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.661182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.661609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] PATH_INFO: `/v3/projects/05802417523e4d33b4792e28755ab498/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.664927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05802417523e4d33b4792e28755ab498, user_id=b0a4e91cd34a4c8a90c8a8ebdf58aa97, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:29.668942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:29.676100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:29.687420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 940/1883] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:29 2022] PUT /identity/v3/projects/05802417523e4d33b4792e28755ab498/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:29.697427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.697725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.697987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.745758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] MFA Rules not processed for user `b0a4e91cd34a4c8a90c8a8ebdf58aa97`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:29.748972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:29.856400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 944/1884] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:29.870884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:29.873080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:29.968031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689e309-9c4b-478b-b271-335537ccdd06 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:29.976569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:29.977489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:29.977987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:29.978454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:29.996465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:30.000274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:30.017217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 941/1885] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:30.241297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:30.242836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:30.272208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2eae44-cc11-452d-90ee-be34d104a0a2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:30.280138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:30.280899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:30.281254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:30.281596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:30.407717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:30.409211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b087cd3-5801-49a0-b61d-7dfc07a57336 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:30.419972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 945/1886] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:38.840486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:38.947491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06528ac-3dd8-4f5c-8c22-e8f403a432bd service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:38.964698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32572de9-b544-429f-b886-37445d2006f8 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:38.965310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32572de9-b544-429f-b886-37445d2006f8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:38.965310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32572de9-b544-429f-b886-37445d2006f8 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:38.967075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:38.967308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 946/1887] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:36:38.968956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:38.969436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:38.969885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] PATH_INFO: `/v3/users/f80229361211489eb5c1dcd49d170cf3` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:38.971693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80229361211489eb5c1dcd49d170cf3)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:38.974378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:38.975898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163277a8-e126-497f-921a-23159536aa00 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:38.976192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163277a8-e126-497f-921a-23159536aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:38.976598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163277a8-e126-497f-921a-23159536aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:39.014051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-163277a8-e126-497f-921a-23159536aa00 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:39.015625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-163277a8-e126-497f-921a-23159536aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:39.081387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80229361211489eb5c1dcd49d170cf3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:39.094388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80229361211489eb5c1dcd49d170cf3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:39.111324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80229361211489eb5c1dcd49d170cf3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:39.112209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80229361211489eb5c1dcd49d170cf3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:39.119832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a4d85355-0413-4d1d-93c2-074bb044aca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80229361211489eb5c1dcd49d170cf3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:39.126065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 942/1888] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:38 2022] DELETE /identity/v3/users/f80229361211489eb5c1dcd49d170cf3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:39.130808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:39.131308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:39.131773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:39.153766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 947/1889] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:39.155068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:39.156338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:39.168129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163277a8-e126-497f-921a-23159536aa00 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:39.169553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-163277a8-e126-497f-921a-23159536aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:39.175470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:39.183431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163277a8-e126-497f-921a-23159536aa00 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:39.196605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:39.229192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 943/1890] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:39.241815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:39.244349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:39.289730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:39.290515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:39.291328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:39.291776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:39.321926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:39.323723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:39.336689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 948/1891] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:39.345335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e946e-049f-4b83-9b8e-b3497ced5681 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:39.350613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:39.351898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:39.352164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:39.352443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:39.364355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:39.366246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:39.381161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 944/1892] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:40.075185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:40.076773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:40.094497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cec15-c3f0-466e-8eac-084ce92f63d7 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:40.109132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:40.110451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:40.111418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:40.113455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] PATH_INFO: `/v3/projects/275d6205355741dca52d516e800aa5c6` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:40.115938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=275d6205355741dca52d516e800aa5c6)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:40.118707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:40.191169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275d6205355741dca52d516e800aa5c6', 'request_id': 'req-67c850e8-77e2-461c-8c81-ef841cf46c7f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.199356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275d6205355741dca52d516e800aa5c6', 'request_id': 'req-67c850e8-77e2-461c-8c81-ef841cf46c7f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.215734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275d6205355741dca52d516e800aa5c6', 'request_id': 'req-67c850e8-77e2-461c-8c81-ef841cf46c7f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.242951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] The token cache is being invalidate because project 275d6205355741dca52d516e800aa5c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:40.243865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.246159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.250892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 949/1893] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:40 2022] DELETE /identity/v3/projects/275d6205355741dca52d516e800aa5c6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:40.257429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:40.284392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06351567-6163-4961-82f1-22a673e32c7b service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:40.395013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:40.396577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:40.396828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:40.397100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] PATH_INFO: `/v3/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:40.400934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a4e91cd34a4c8a90c8a8ebdf58aa97)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:40.406415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:40.469463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:40.472739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:40.486219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a4e91cd34a4c8a90c8a8ebdf58aa97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.500171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a4e91cd34a4c8a90c8a8ebdf58aa97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.501924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c850e8-77e2-461c-8c81-ef841cf46c7f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:40.508871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:40.509839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:40.510771 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:40.511144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] PATH_INFO: `/v3/users/38dc28d556424f539cdc8c4b67fdb996` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:40.512553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38dc28d556424f539cdc8c4b67fdb996)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:40.515730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:40.516011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:40.517794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a4e91cd34a4c8a90c8a8ebdf58aa97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.518608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a4e91cd34a4c8a90c8a8ebdf58aa97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.527930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-daf8e8a8-d500-4b50-bf27-6ebf0b908e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a4e91cd34a4c8a90c8a8ebdf58aa97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.535949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 945/1894] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:40 2022] DELETE /identity/v3/users/b0a4e91cd34a4c8a90c8a8ebdf58aa97 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:40.583577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38dc28d556424f539cdc8c4b67fdb996'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.606446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38dc28d556424f539cdc8c4b67fdb996'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.607576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38dc28d556424f539cdc8c4b67fdb996'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.633893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38dc28d556424f539cdc8c4b67fdb996'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.642822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:40.646643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38dc28d556424f539cdc8c4b67fdb996'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:40.658863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 950/1895] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:40 2022] DELETE /identity/v3/users/38dc28d556424f539cdc8c4b67fdb996 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:40.665475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:40.665996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:40.666993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:40.691807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:40.694008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:40.853043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 946/1896] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:40.882055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:40.882736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:41.005854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7751-7ddf-4d71-a3ec-a19a821c95a7 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.016874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.018244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.018574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.018803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.046848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.051414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad30fb-ffb1-492d-bf8a-587ab25733f8 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.078618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 951/1897] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:41.134509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.136330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:41.152875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583da31-15ee-4c39-a2ae-72f91255cdf6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.159392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.160498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.160785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.161057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] PATH_INFO: `/v3/projects/05802417523e4d33b4792e28755ab498` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.163924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=05802417523e4d33b4792e28755ab498)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.166730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.230761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05802417523e4d33b4792e28755ab498', 'request_id': 'req-7a5dca91-8478-4a96-9bad-3af5998bbfbf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:41.241371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05802417523e4d33b4792e28755ab498', 'request_id': 'req-7a5dca91-8478-4a96-9bad-3af5998bbfbf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:41.249967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05802417523e4d33b4792e28755ab498', 'request_id': 'req-7a5dca91-8478-4a96-9bad-3af5998bbfbf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:41.268291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] The token cache is being invalidate because project 05802417523e4d33b4792e28755ab498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:41.268958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:41.269600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7a5dca91-8478-4a96-9bad-3af5998bbfbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:41.273911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 947/1898] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:41 2022] DELETE /identity/v3/projects/05802417523e4d33b4792e28755ab498 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:41.279637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.280205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.280625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.301610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:41.303599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:41.376454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 952/1899] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:41.390289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.390689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.391008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.414033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:41.415352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:41.424499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 948/1900] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:41.429936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.446036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950ac0e-1782-41dd-8ebf-b56a4fb3494e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.454404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.455327 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.456219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.456219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.457245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:41.457645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.460632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.478497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 953/1901] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:41.480085 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.490869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b154d8-629a-4394-b034-5f80f287f360 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.507870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.508811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.509642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.510088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.511163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.513864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.557480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 949/1902] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:41 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:41.561767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.574522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c371977-40a5-4fb2-9840-b2e84bec2765 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.656294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.657199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.657602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.658106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.659189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.662047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.694680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 954/1903] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:41 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:41.696216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.712016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28051a2-e5f3-4a92-a52c-4b1a01fd1fbe admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.725428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.729016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.729927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.730365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.731280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.733427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.747606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 950/1904] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:41.752310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.770295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f0aec-8b68-4ba5-be8b-d064d22a6d65 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.778335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.781677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.781677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.781677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.781677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.782889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.814079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 955/1905] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:41.819456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.837885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32fc5b-2326-4042-9905-241b9c798a09 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.842524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:41.844546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:41.860630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.861707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.863911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.864402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] PATH_INFO: `/v3/projects/247964948a3e4c329281af9aec0ec2c2/users/d15ad4370c754642a009b97fe6326670/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.867513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:41.874761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca113cc-2c55-4d6d-bdb7-0399ee87c7f8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:41.883639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:41.884552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.884922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.885208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] PATH_INFO: `/v3/projects/e410e4c56d7347419d7784bf56294527` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.887692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e410e4c56d7347419d7784bf56294527)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.889872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.915736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247964948a3e4c329281af9aec0ec2c2, user_id=d15ad4370c754642a009b97fe6326670, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:41.919046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56772a82-4324-4b49-be03-eac724a36d32 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:41.942965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 951/1906] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:41 2022] PUT /identity/v3/projects/247964948a3e4c329281af9aec0ec2c2/users/d15ad4370c754642a009b97fe6326670/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:41.957861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:41.959677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:41.960865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:41.974671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:41.982716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e410e4c56d7347419d7784bf56294527', 'request_id': 'req-57ba2995-f011-4b2f-90b5-43a007bf7e59'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.007392 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e410e4c56d7347419d7784bf56294527', 'request_id': 'req-57ba2995-f011-4b2f-90b5-43a007bf7e59'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.011169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] MFA Rules not processed for user `d15ad4370c754642a009b97fe6326670`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:42.013371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.022756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e410e4c56d7347419d7784bf56294527', 'request_id': 'req-57ba2995-f011-4b2f-90b5-43a007bf7e59'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.053461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] The token cache is being invalidate because project e410e4c56d7347419d7784bf56294527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:42.055639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.057731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.076553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 956/1907] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:41 2022] DELETE /identity/v3/projects/e410e4c56d7347419d7784bf56294527 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:42.078635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.092797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.188655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 952/1908] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:42.195351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.197721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.238096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba2995-f011-4b2f-90b5-43a007bf7e59 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.246513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.247476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.248102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.248295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] PATH_INFO: `/v3/users/a5508783f8494fedbbf072cbfed355da` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.250239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5508783f8494fedbbf072cbfed355da)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.253410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.303759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cacbbfb-3fad-4e18-aaa5-fe58019e8e3f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.309744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.310853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.311144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.311511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] PATH_INFO: `/v3/users/511e410ce9404196aad56a7cad98855b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.313029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=511e410ce9404196aad56a7cad98855b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.315260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.334574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5508783f8494fedbbf072cbfed355da'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.344090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5508783f8494fedbbf072cbfed355da'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.344848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5508783f8494fedbbf072cbfed355da'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.360653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5508783f8494fedbbf072cbfed355da'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.369409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5508783f8494fedbbf072cbfed355da'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.376316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 957/1909] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:42 2022] DELETE /identity/v3/users/a5508783f8494fedbbf072cbfed355da => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:42.377198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.378944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.379434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '511e410ce9404196aad56a7cad98855b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.393735 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:42.394494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '511e410ce9404196aad56a7cad98855b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.417164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '511e410ce9404196aad56a7cad98855b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.417978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '511e410ce9404196aad56a7cad98855b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.427657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-130c7434-d6a1-40b6-b99a-30f687fbe941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '511e410ce9404196aad56a7cad98855b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:42.434429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 953/1910] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:42 2022] DELETE /identity/v3/users/511e410ce9404196aad56a7cad98855b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:42.439939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.440235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.440523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.472459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:42.473785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.497403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:42.531417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae0-6baf-4df8-8b14-126a00261fb8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.571990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 954/1911] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:42.584727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.586762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.602806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.603654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.603927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.604198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.604896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.606915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.635329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 958/1912] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:42 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:42.639589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.642750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.655606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadb6af-001a-433c-9fc3-b032a20226a7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.720828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.721502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.721787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.723642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.726277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46427bf8-3b52-4f5b-8f88-e97292e2e034 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.732332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.733082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.733483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.733863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.735012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.739019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.740788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.742290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.772863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 955/1913] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:42.784694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 959/1914] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:42 2022] POST /identity/v3/users => generated 297 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:42.789826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.802082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e56c0-1f09-4f45-b56d-13592ee8f117 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.816170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.817076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.818109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.818568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.820127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.822853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.842652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 956/1915] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:42.844361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.856539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0e7a-6a4d-4ace-afb8-95ea95c7eebc admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.864661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.865850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.866191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.866466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] PATH_INFO: `/v3/projects/267862bc471b48599041b2f347084b0b/users/df28ce840b644b3b9570ef12da4faa63/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.879188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.884296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:42.909546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b1389-51f7-47e0-b5d3-f472e1c8d8f0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:42.916614 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:42.917990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:42.922366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=267862bc471b48599041b2f347084b0b, user_id=df28ce840b644b3b9570ef12da4faa63, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.924684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:42.925013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] PATH_INFO: `/v3/projects/615933bca2cc471d86223ce5a686fa5a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:42.925199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.927755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615933bca2cc471d86223ce5a686fa5a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:42.933001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:42.945586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 960/1916] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:42 2022] PUT /identity/v3/projects/267862bc471b48599041b2f347084b0b/users/df28ce840b644b3b9570ef12da4faa63/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:42.949222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:42.961144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:43.010737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615933bca2cc471d86223ce5a686fa5a', 'request_id': 'req-a96db507-847f-465e-996a-5db03e5eca97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.024415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615933bca2cc471d86223ce5a686fa5a', 'request_id': 'req-a96db507-847f-465e-996a-5db03e5eca97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.029885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:43.040101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615933bca2cc471d86223ce5a686fa5a', 'request_id': 'req-a96db507-847f-465e-996a-5db03e5eca97'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.065761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] The token cache is being invalidate because project 615933bca2cc471d86223ce5a686fa5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:43.066052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.067515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a96db507-847f-465e-996a-5db03e5eca97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.073518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 957/1917] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:42 2022] DELETE /identity/v3/projects/615933bca2cc471d86223ce5a686fa5a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:43.075197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5162-c787-4549-8645-347b089f0277 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.075528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5162-c787-4549-8645-347b089f0277 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.075808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b5162-c787-4549-8645-347b089f0277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.085312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31929-bb30-4d43-bd93-e15ddf43e428 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.099643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1f8b5162-c787-4549-8645-347b089f0277 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:43.101516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1f8b5162-c787-4549-8645-347b089f0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.194344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 958/1918] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:43.195727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.196041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.196322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.216893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:43.218468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.228190 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 959/1919] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:43.232847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.237956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.238854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.240775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.241184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.242412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.244222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.244582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b747b03-9ad4-42e1-a85d-5f7ef099c07d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.250839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.250839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.251529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.251658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.252496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:43.253143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.254825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.257972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 961/1920] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:43.264186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.282468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.295504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 960/1921] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:36:43.297757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.312454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ea188-5ed4-485c-8c94-66c05f1ef30a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.313495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bfd69-49b3-4cbb-a9d1-8d3812e7ba23 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.319920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.320128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.320417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.320692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.320799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.321533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.322054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.322425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.322907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] PATH_INFO: `/v3/projects/267862bc471b48599041b2f347084b0b/users/df28ce840b644b3b9570ef12da4faa63/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.323833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.326149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=267862bc471b48599041b2f347084b0b, user_id=df28ce840b644b3b9570ef12da4faa63, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.331005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff41b9-7c0a-4478-bb94-baca2539dcdf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.352925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 962/1922] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:43 2022] PUT /identity/v3/projects/267862bc471b48599041b2f347084b0b/users/df28ce840b644b3b9570ef12da4faa63/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:43.363603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.365278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.365278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.371357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 961/1923] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:43 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:43.377317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.402503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] MFA Rules not processed for user `df28ce840b644b3b9570ef12da4faa63`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:43.403957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.491135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a59cab-4bc4-42a7-831c-8f9e5cfb4ae2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.497509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.498324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.498728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.499101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.500132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.504440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.545012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 962/1924] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:43 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:43.546939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 963/1925] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:43.548750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.555433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.561713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.576479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b7b59-0c81-44a5-a7a8-c97df09dbaaf None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.590387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.591384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.591674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.591954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.592704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.594316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.598666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643c692-eb0e-4abc-8b69-1585a7ee8934 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.612681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.613703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.613985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.614258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] PATH_INFO: `/v3/projects/b8f8cdc280f34f64a14b813e91bc11b7` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.621156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 964/1926] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:43.621156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.621407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f8cdc280f34f64a14b813e91bc11b7)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.623489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.624389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.637007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:43.709567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:43.715100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ddd15-e67a-45a6-b649-e8b3363bdaab admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.733679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f8cdc280f34f64a14b813e91bc11b7', 'request_id': 'req-39599e46-81be-4a46-b00a-8a882dc45c0f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.747831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f8cdc280f34f64a14b813e91bc11b7', 'request_id': 'req-39599e46-81be-4a46-b00a-8a882dc45c0f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.755830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f8cdc280f34f64a14b813e91bc11b7', 'request_id': 'req-39599e46-81be-4a46-b00a-8a882dc45c0f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.770964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] The token cache is being invalidate because project b8f8cdc280f34f64a14b813e91bc11b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:43.771637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.772508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:43.774466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 963/1927] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:43 2022] DELETE /identity/v3/projects/b8f8cdc280f34f64a14b813e91bc11b7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:43.775412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.776778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.781258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.782321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.782971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.783478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.785776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.870949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.873026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.887609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 965/1928] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:43.888868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:43.890401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:43.922867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39599e46-81be-4a46-b00a-8a882dc45c0f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.942647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.943853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.944143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.944481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.945167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.951460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:43.987829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e4c4e-90a5-4f3f-8f2c-871e0c229422 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:43.994125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:43.995099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:43.995413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:43.995716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] PATH_INFO: `/v3/users/5a0645168d834c0aa0f2ab1c06af8d65` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:43.997173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0645168d834c0aa0f2ab1c06af8d65)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:43.999334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.009711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 964/1929] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:44.011723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.018606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:44.046554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78035949-3a41-4274-9bc8-297e652eb488 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.061059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:44.062153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.062450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.062732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] PATH_INFO: `/v3/projects/0a2e109e63e0438198d548c238c0a2ce/users/7fb745ee044a4675bad8cde1252dfd95/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.081708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0645168d834c0aa0f2ab1c06af8d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.093799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0645168d834c0aa0f2ab1c06af8d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.094530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2e109e63e0438198d548c238c0a2ce, user_id=7fb745ee044a4675bad8cde1252dfd95, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:44.094654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0645168d834c0aa0f2ab1c06af8d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.098049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509b7a-bdf9-4c4a-987f-2b9af9aa239d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.119415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 965/1930] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:44 2022] PUT /identity/v3/projects/0a2e109e63e0438198d548c238c0a2ce/users/7fb745ee044a4675bad8cde1252dfd95/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:44.129331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.129755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.130144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0645168d834c0aa0f2ab1c06af8d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.130399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.150538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0645168d834c0aa0f2ab1c06af8d65'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.168263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 966/1931] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:43 2022] DELETE /identity/v3/users/5a0645168d834c0aa0f2ab1c06af8d65 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:44.227193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] MFA Rules not processed for user `7fb745ee044a4675bad8cde1252dfd95`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:44.236592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.395959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 966/1932] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:44.397798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.399413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.496640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33752c86-6484-40c1-8fd1-fd70f371ac1a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.503172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:44.504106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.504482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.504844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.505828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:44.508797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.529373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.531053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.548965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcce8ee-e632-4b69-abcf-3655aa880e36 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.551552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 967/1933] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:44 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:44.556193 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.567848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:44.624858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:44.633717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.633717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.633717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] PATH_INFO: `/v3/users/d15ad4370c754642a009b97fe6326670` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.633717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15ad4370c754642a009b97fe6326670)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:44.633717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.651442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c4a31-4667-490e-bffc-b51986b47ec6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.656595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:44.657369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.657659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.657942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.658862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:44.661417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.686050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 968/1934] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:44 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:44.687075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.689542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.703866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15ad4370c754642a009b97fe6326670'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.706313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbf6f0-7d80-4ffa-b893-87023daaa99c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.713302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:44.714127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15ad4370c754642a009b97fe6326670'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.714239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.714521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.714835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] PATH_INFO: `/v3/projects/3bbe532467e5412ba80fb11b2d8b820f` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.716358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bbe532467e5412ba80fb11b2d8b820f)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:44.718638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:44.721923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:44.741451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15ad4370c754642a009b97fe6326670'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.742252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15ad4370c754642a009b97fe6326670'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.754544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15ad4370c754642a009b97fe6326670'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.776611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 967/1935] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:44 2022] DELETE /identity/v3/users/d15ad4370c754642a009b97fe6326670 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:44.781302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:44.782769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.807869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bbe532467e5412ba80fb11b2d8b820f', 'request_id': 'req-ff364540-fc39-4681-b026-7ecc6d0f9161'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.817860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bbe532467e5412ba80fb11b2d8b820f', 'request_id': 'req-ff364540-fc39-4681-b026-7ecc6d0f9161'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.829572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bbe532467e5412ba80fb11b2d8b820f', 'request_id': 'req-ff364540-fc39-4681-b026-7ecc6d0f9161'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.865113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] The token cache is being invalidate because project 3bbe532467e5412ba80fb11b2d8b820f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:44.865425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.866703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ff364540-fc39-4681-b026-7ecc6d0f9161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:44.868615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 969/1936] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:44 2022] DELETE /identity/v3/projects/3bbe532467e5412ba80fb11b2d8b820f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:44.873981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d32b-23fd-465d-8e1d-70adb983a85f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.874304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d32b-23fd-465d-8e1d-70adb983a85f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.874738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4d32b-23fd-465d-8e1d-70adb983a85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:44.896926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-02d4d32b-23fd-465d-8e1d-70adb983a85f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:44.898168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-02d4d32b-23fd-465d-8e1d-70adb983a85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:44.912161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853077c-93a5-405b-845e-116f2d9a0896 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:44.981733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 970/1937] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:44.981733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:44.981733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:44.981733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.022696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:45.034402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:45.041046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.043915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.044604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.044888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.045168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.046051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.047957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.063932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 968/1938] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:45.065666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.068104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 971/1939] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:45.068490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.069517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.076793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.104538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f2433-6052-4735-9c5c-88471ac48bba None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.113194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.114440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.114738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.115013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] PATH_INFO: `/v3/projects/6adae6e6c55d45a7848b88a3a55991b8/users/4f650b9b7dd041c0a6416b823e1f6a1a/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.133743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adae6e6c55d45a7848b88a3a55991b8, user_id=4f650b9b7dd041c0a6416b823e1f6a1a, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.136579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b87e8-1981-4c80-94eb-fc983a567983 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.151950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95b8d8-63a2-4573-94b2-469dbef03ab0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.158557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 972/1940] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:45 2022] PUT /identity/v3/projects/6adae6e6c55d45a7848b88a3a55991b8/users/4f650b9b7dd041c0a6416b823e1f6a1a/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:45.160164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.161080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.161080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.191153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:45.194261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.224370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.225078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.225516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.226086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.313137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 973/1941] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:45.317868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.323971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:45.332283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003847c3-d34a-4f17-a2a1-8e028dd3416f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.339597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.341767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.342404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.343244 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.343963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.344694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.346101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.349525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.359209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 969/1942] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:45.360129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.372178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8654559-adf2-4dad-81a9-a812b875c357 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.377089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 974/1943] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:45.378119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.378870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.379403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.379865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.380689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:45.381117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.382805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.383505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.397669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735501f-6910-447d-a55e-395d08e6d425 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.407865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.409981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.410419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.410888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] PATH_INFO: `/v3/projects/6adae6e6c55d45a7848b88a3a55991b8/users/4f650b9b7dd041c0a6416b823e1f6a1a/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.414097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 970/1944] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:45.419636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.420611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6adae6e6c55d45a7848b88a3a55991b8, user_id=4f650b9b7dd041c0a6416b823e1f6a1a, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.425053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87d4af-ab2c-468e-b727-d0361eed803a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.453500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff727a-771b-4a31-bc62-bae24f57a8a5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.455472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 975/1945] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:45 2022] PUT /identity/v3/projects/6adae6e6c55d45a7848b88a3a55991b8/users/4f650b9b7dd041c0a6416b823e1f6a1a/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:45.457951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.459001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.459516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.510281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:45.514823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] MFA Rules not processed for user `4f650b9b7dd041c0a6416b823e1f6a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:45.519106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.570517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.572270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.575243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.575243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.576013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.579207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.685895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 971/1946] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:45 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:45.691746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.733449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 976/1947] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:45.743594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.749944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.844767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd726bd7-df01-4751-bf26-4307b5eeb7fa admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.852065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178718a6-8b35-48ba-a12e-75a752e45a02 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.853119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.853816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.854100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.854372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.855066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.857057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.886418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 972/1948] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:45 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:45.888642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:45.890656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:45.912661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7de8-a717-4428-9944-3f335012847a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:45.918806 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.919743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.920029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.920305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] PATH_INFO: `/v3/projects/247964948a3e4c329281af9aec0ec2c2` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.921606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] RBAC: Authorizing `identity:delete_project(project_id=247964948a3e4c329281af9aec0ec2c2)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:45.923602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:45.954077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:45.954756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:45.955059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:45.955636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:45.987558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247964948a3e4c329281af9aec0ec2c2', 'request_id': 'req-d43e33ff-a25e-4cb6-86a3-889b520bacce'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:45.997511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:45.998079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247964948a3e4c329281af9aec0ec2c2', 'request_id': 'req-d43e33ff-a25e-4cb6-86a3-889b520bacce'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.005722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247964948a3e4c329281af9aec0ec2c2', 'request_id': 'req-d43e33ff-a25e-4cb6-86a3-889b520bacce'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.021855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] The token cache is being invalidate because project 247964948a3e4c329281af9aec0ec2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:46.022245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.023012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.024790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 973/1949] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:45 2022] DELETE /identity/v3/projects/247964948a3e4c329281af9aec0ec2c2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:46.025692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.028547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:46.035623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.037690 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.050457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 977/1950] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:46.051747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.072515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a43b0-086d-443d-b13d-d0aa826c0c9d service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:46.095676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:46.169313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:46.170452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.170736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.171024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] PATH_INFO: `/v3/users/df28ce840b644b3b9570ef12da4faa63` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.173150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df28ce840b644b3b9570ef12da4faa63)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.175430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.179344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e33ff-a25e-4cb6-86a3-889b520bacce admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:46.187738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:46.188417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.188699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.188972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.189698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.192238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.214535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 974/1951] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:46.221052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.271832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df28ce840b644b3b9570ef12da4faa63'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.274314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a31fb-49f7-46a8-b5ba-f049dd3e7317 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:46.281188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:46.282218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df28ce840b644b3b9570ef12da4faa63'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.282761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.282871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df28ce840b644b3b9570ef12da4faa63'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.283450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.284151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.285754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.288210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.298062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df28ce840b644b3b9570ef12da4faa63'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.313412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 975/1952] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:46.319737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.321658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-4a95c86d-c4bf-40ce-9c35-b65257ddd395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df28ce840b644b3b9570ef12da4faa63'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:46.323991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:46.331903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 978/1953] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:46 2022] DELETE /identity/v3/users/df28ce840b644b3b9570ef12da4faa63 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:46.346117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20537ecc-ea85-4d14-8608-4e9dc3e141a6 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:46.437707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:46.438861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.439059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.439378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] PATH_INFO: `/v3/projects/575fd53c944e40f4a0ba834c28c73d52/users/f3312ef8ca6f455e87fbbbfd0b9a9664/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.460676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575fd53c944e40f4a0ba834c28c73d52, user_id=f3312ef8ca6f455e87fbbbfd0b9a9664, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.463596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.482295 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 976/1954] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:46 2022] PUT /identity/v3/projects/575fd53c944e40f4a0ba834c28c73d52/users/f3312ef8ca6f455e87fbbbfd0b9a9664/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:46.495159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.496403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.496959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.561133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] MFA Rules not processed for user `f3312ef8ca6f455e87fbbbfd0b9a9664`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:46.564012 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:46.664214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:46.713531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 979/1955] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:46.718116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.804575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.806004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:46.835676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07f341-32ec-4e91-85f9-0139fe30b265 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:46.841988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:46.843509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:46.847394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:46.847744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:46.848468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:46.852044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:46.893832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 977/1956] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:46 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:46.899249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:46.942646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:46.949924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dce76-feca-42e3-8c91-3af7f39f8af6 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.034724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9128b4-73a2-44d9-b35f-64788ef96716 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.053053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.053053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.053053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.053053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.053530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.053530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.096129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 978/1957] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:46 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:47.100767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.102277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.103161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.103471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.108064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] PATH_INFO: `/v3/projects/267862bc471b48599041b2f347084b0b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.110588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] RBAC: Authorizing `identity:delete_project(project_id=267862bc471b48599041b2f347084b0b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.112539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.119657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67ca60-c2e1-4441-8417-5ec367f2e896 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.136960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.139011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.139438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.141620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.142841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.146831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.167224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 979/1958] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:47.172383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.188702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746356f6-3a69-46b1-9920-6162938083ee admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.220147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267862bc471b48599041b2f347084b0b', 'request_id': 'req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:47.235560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267862bc471b48599041b2f347084b0b', 'request_id': 'req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:47.255786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267862bc471b48599041b2f347084b0b', 'request_id': 'req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:47.281078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] The token cache is being invalidate because project 267862bc471b48599041b2f347084b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:47.282245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:47.283111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1043b4c5-8d55-4d39-9b5e-7e2524f0aead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:47.288467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 980/1959] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:46 2022] DELETE /identity/v3/projects/267862bc471b48599041b2f347084b0b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:47.297051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ee42-b7e3-4635-95a0-cf7530b633cc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.297345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ee42-b7e3-4635-95a0-cf7530b633cc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.297644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ee42-b7e3-4635-95a0-cf7530b633cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.332599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.333587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.334045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.335024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.340686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.342799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.353724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-9130ee42-b7e3-4635-95a0-cf7530b633cc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:47.362208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9130ee42-b7e3-4635-95a0-cf7530b633cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:47.362767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 980/1960] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:47.363836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.366791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:47.384782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f84be-b470-42de-9b53-3258dce59502 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.398655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 981/1961] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:47.402102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.404279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.404652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.404934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] PATH_INFO: `/v3/projects/a0217eb4d9b947edac57aa034adeb4fa/users/ee925e3bd2654f4e97022232f68985b5/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.409468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.410044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.410536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.443556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:47.444940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:47.466160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 982/1962] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:47.467333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.483391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0217eb4d9b947edac57aa034adeb4fa, user_id=ee925e3bd2654f4e97022232f68985b5, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.484067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:47.494494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e2a7e-99d0-4908-8915-4b341a2bd1ce admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.501433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d3bf-052c-4880-a3e4-5c468a115854 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.515418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.516256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.517267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.517652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.518681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:47.519383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.526353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.530899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 981/1963] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:47 2022] PUT /identity/v3/projects/a0217eb4d9b947edac57aa034adeb4fa/users/ee925e3bd2654f4e97022232f68985b5/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:47.546967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.547446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.547876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.574998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 983/1964] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:36:47.576102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.590292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fdc2b-0fc5-415f-8fe9-a150bf7718c6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.592714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] MFA Rules not processed for user `ee925e3bd2654f4e97022232f68985b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:47.595211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:47.770270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.773499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.773883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.774223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.775025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.778687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.809240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 982/1965] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:47.813877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.826874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:47.845334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 984/1966] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:47.847098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b889a3-b0f9-4241-9762-8f380877a864 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.850171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:47.969553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbf277-35f6-4a07-91de-6e40e69a65e8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:47.970550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.971189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.972472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.972757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.973479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.976345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:47.977320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:47.978338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:47.979758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:47.980147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:47.982518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:47.984843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.015331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 983/1967] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:48.015491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 985/1968] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:47 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:48.019137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.019321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.032914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a429883-d751-41cc-9697-625518fe903f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.096591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd05e-a7fb-4dce-8cbb-65d78eeb6074 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.097534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.098347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.098638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.098972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.100103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.101821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.102836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.103594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.103878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.104157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.104947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.107116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.107878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:48.119143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 986/1969] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:48.123796 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.133732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 984/1970] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:48 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:48.137319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.138225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e37b-8c46-438f-8439-5677a8a92fd6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.143191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.144008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.144413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.144801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.146027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.148024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.149082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b95584-62d2-4c1c-bbd4-e6284920c4e3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.156248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.156928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.157371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.157840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.158795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.161112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.167090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 987/1971] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:48.168515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.174095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 985/1972] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:48.179131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.181055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3607-2436-4e8a-b5cd-017710a77344 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.189598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.190458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd26eea-87f8-4664-ba95-a9c361179bd7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.190744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.191102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.191402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] PATH_INFO: `/v3/projects/0fd6036befbe4dfd91e017249d5b0b57/users/d77ebd7fa57940edaab782c20e031f6a/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.196640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.197647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.198003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.198346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] PATH_INFO: `/v3/projects/96b458659b6940f2b85076517a807045/users/9a22427801ca4b7d966b030e66c7ccb8/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.211395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd6036befbe4dfd91e017249d5b0b57, user_id=d77ebd7fa57940edaab782c20e031f6a, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.214257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae42cc-b868-4717-8be3-1d67f751268d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.216128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b458659b6940f2b85076517a807045, user_id=9a22427801ca4b7d966b030e66c7ccb8, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.219119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.231326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 988/1973] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:48 2022] PUT /identity/v3/projects/0fd6036befbe4dfd91e017249d5b0b57/users/d77ebd7fa57940edaab782c20e031f6a/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:48.235847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 986/1974] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:48 2022] PUT /identity/v3/projects/96b458659b6940f2b85076517a807045/users/9a22427801ca4b7d966b030e66c7ccb8/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:48.243278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.243685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.243797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.244014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.276673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] MFA Rules not processed for user `d77ebd7fa57940edaab782c20e031f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:48.278161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.310871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:48.335738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5669140-241c-456d-8241-f11d214e8eef admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.342587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.343556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.343847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.344673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.345698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.347542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.362669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 987/1975] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:48.364616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 989/1976] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:48.367179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.376623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.378226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.378478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789df73-f7d8-4c11-a0fa-cf1fc654a8e8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.383975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.385042 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.385985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.386124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] PATH_INFO: `/v3/projects/96b458659b6940f2b85076517a807045/users/9a22427801ca4b7d966b030e66c7ccb8/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.388448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b458659b6940f2b85076517a807045, user_id=9a22427801ca4b7d966b030e66c7ccb8, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.391147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c732ca-b932-4298-b4f9-6a3a21a8e63e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.410897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 988/1977] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:48 2022] PUT /identity/v3/projects/96b458659b6940f2b85076517a807045/users/9a22427801ca4b7d966b030e66c7ccb8/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:48.421321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.421616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.421904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.449056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680c53-62e6-4e61-a49d-261b56ae7f3f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.473039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] MFA Rules not processed for user `9a22427801ca4b7d966b030e66c7ccb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:48.474683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.543633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.544295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.544574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.544847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.566762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 989/1978] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:48.566762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.566762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.578982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:48.611790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.613144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.623404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 990/1979] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:48.626059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.627430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.635316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bc950-dff3-46c7-a9b0-d1d660005a57 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.639282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae34357-4693-4b00-bf50-8f410b8fdd78 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:48.640299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.641165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.641449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.641725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] PATH_INFO: `/v3/users/7fb745ee044a4675bad8cde1252dfd95` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.642920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb745ee044a4675bad8cde1252dfd95)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.643421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:48.644143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.644487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.644828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.644939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.655278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:48.656879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517df97e-1e27-44d4-a624-75ac89332338 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:48.668243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 991/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:48.709114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb745ee044a4675bad8cde1252dfd95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:48.718370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb745ee044a4675bad8cde1252dfd95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:48.720113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:48.743039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb745ee044a4675bad8cde1252dfd95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:48.744123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb745ee044a4675bad8cde1252dfd95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:48.751826 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb745ee044a4675bad8cde1252dfd95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:48.758411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 990/1981] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:48 2022] DELETE /identity/v3/users/7fb745ee044a4675bad8cde1252dfd95 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:48.762615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:48.762903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:48.763181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:48.809732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:48.811208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:48.948527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 992/1982] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:48.956702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:48.961193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:49.069932 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42eef1-a6c0-4fc1-b838-75f6dfc7801a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:49.073969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:49.075117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:49.075439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:49.075763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:49.087563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:49.090121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:49.100854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 991/1983] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:49.606659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:49.608501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:49.676544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad12a8-abe1-449c-af02-7fb5f11720b9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:49.681426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:49.681676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:49.682074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:49.682524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:49.684974 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:49.744778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:49.746201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:49.772473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035da0-df70-49a5-8235-424b69a9c185 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:49.785916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:49.786864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:49.787154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:49.787465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] PATH_INFO: `/v3/projects/0a2e109e63e0438198d548c238c0a2ce` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:49.793150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2e109e63e0438198d548c238c0a2ce)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:49.795164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:49.817531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:49.819897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:49.829799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 993/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:49.891498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2e109e63e0438198d548c238c0a2ce', 'request_id': 'req-4f18ea53-1c38-44d0-b01c-02de23ecdac8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:49.905371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2e109e63e0438198d548c238c0a2ce', 'request_id': 'req-4f18ea53-1c38-44d0-b01c-02de23ecdac8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:49.913968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2e109e63e0438198d548c238c0a2ce', 'request_id': 'req-4f18ea53-1c38-44d0-b01c-02de23ecdac8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:49.928883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] The token cache is being invalidate because project 0a2e109e63e0438198d548c238c0a2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:49.929534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:49.930781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:49.933369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 992/1985] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:49 2022] DELETE /identity/v3/projects/0a2e109e63e0438198d548c238c0a2ce => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:49.937288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:49.950920 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389254-624b-4200-b0e7-5e892a953839 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:49.956426 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:50.023480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:50.024517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:50.024944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:50.025282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] PATH_INFO: `/v3/users/4f650b9b7dd041c0a6416b823e1f6a1a` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:50.026548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f650b9b7dd041c0a6416b823e1f6a1a)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:50.028599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:50.155935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f650b9b7dd041c0a6416b823e1f6a1a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:50.166797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f650b9b7dd041c0a6416b823e1f6a1a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:50.167855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f650b9b7dd041c0a6416b823e1f6a1a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:50.184849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f650b9b7dd041c0a6416b823e1f6a1a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:50.193928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e5d1d484-59c8-49b0-a8f1-e8fe672e2ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f650b9b7dd041c0a6416b823e1f6a1a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:50.207682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 994/1986] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:49 2022] DELETE /identity/v3/users/4f650b9b7dd041c0a6416b823e1f6a1a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:50.888037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:50.890894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:50.908700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18ea53-1c38-44d0-b01c-02de23ecdac8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:50.989862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:50.998120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:50.999360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.000007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.000489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] PATH_INFO: `/v3/projects/6adae6e6c55d45a7848b88a3a55991b8` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.001897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6adae6e6c55d45a7848b88a3a55991b8)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.003915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.054390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6adae6e6c55d45a7848b88a3a55991b8', 'request_id': 'req-a62ca905-2050-49ed-a196-92f2d8806874'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:51.061281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6adae6e6c55d45a7848b88a3a55991b8', 'request_id': 'req-a62ca905-2050-49ed-a196-92f2d8806874'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:51.067392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6adae6e6c55d45a7848b88a3a55991b8', 'request_id': 'req-a62ca905-2050-49ed-a196-92f2d8806874'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:51.078980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] The token cache is being invalidate because project 6adae6e6c55d45a7848b88a3a55991b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:51.079268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:51.079942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a62ca905-2050-49ed-a196-92f2d8806874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:51.081761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 993/1987] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:50 2022] DELETE /identity/v3/projects/6adae6e6c55d45a7848b88a3a55991b8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:51.086752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.087310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.087310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.111718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:51.113019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:51.176371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 995/1988] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:51.184726 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.185039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.185339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.203974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:51.205193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:51.213862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 994/1989] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:51.218233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.228635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b99a7a4-a863-42aa-94e9-3864ba761a1f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.233340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.233999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.234284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.234561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.235304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:51.235627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.237225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.249670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 996/1990] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:51.253691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.263390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf81a6-4761-4ebf-9b2b-3ddc7dc22c8c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.268587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.269248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.269560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.269803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.270512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.272487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.299387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 995/1991] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:51.303673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.313863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30341d5c-4cbd-4c79-8cfe-fd1cdb3dc90a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.323968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:51.388205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.388905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.389174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.389450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.390283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.392375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.419364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 997/1992] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:51.422348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.450899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05e626-7e36-47ee-959c-d7744bc9e0a8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.456311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.457070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.457444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.457725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.458634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.460301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.472640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 996/1993] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:51.477278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.491507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b9b82-d48b-4410-8edd-278bfcb16532 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.496523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.497202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.497540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.497825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.498707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.500507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce18f1-8c31-40b5-8b6a-1b60ac3bc4ef admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.512737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 998/1994] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:51.520289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.531096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1a918-9669-4828-9f76-8c5ba073520b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.537034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.538011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.538302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.538579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] PATH_INFO: `/v3/projects/38128058425e4bb3ac5e5f8b4690875e/users/dcb7104515774edcb36ebd09850eb609/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.567420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38128058425e4bb3ac5e5f8b4690875e, user_id=dcb7104515774edcb36ebd09850eb609, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.571842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.590743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 997/1995] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:51 2022] PUT /identity/v3/projects/38128058425e4bb3ac5e5f8b4690875e/users/dcb7104515774edcb36ebd09850eb609/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:51.601347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.601774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.602146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.629211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] MFA Rules not processed for user `dcb7104515774edcb36ebd09850eb609`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:51.631054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:51.699246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 999/1996] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:51.704358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.785235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe2f4c-5925-4625-b416-add852e6bc51 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.791443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.792214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.792570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.792921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.793693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.795991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.832463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 998/1997] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:51.834164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.846022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e225861-ef7c-4b8a-ba82-b28c16d7c6e8 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.935858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.936809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.937092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.937370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.938072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.940241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.963818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1000/1998] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:51 2022] POST /identity/v3/users => generated 300 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:51.967636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:51.979478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe8c5c-832c-4ef2-ae66-7e1ae40e6938 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:51.985108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:51.985869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:51.987529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:51.987529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:51.987529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:51.988973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:51.992698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:52.014000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 999/1999] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:52.021014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.035474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714db7-bf2e-4ac5-a279-803a176c46a5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.045390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.046493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.046853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.047204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] PATH_INFO: `/v3/projects/18942b14df804b998a3342ad2fe18ab6/users/b9cd204bb43041da82b2be7b9df878fe/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.057489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:52.070443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18942b14df804b998a3342ad2fe18ab6, user_id=b9cd204bb43041da82b2be7b9df878fe, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.073490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.092217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1001/2000] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:52 2022] PUT /identity/v3/projects/18942b14df804b998a3342ad2fe18ab6/users/b9cd204bb43041da82b2be7b9df878fe/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:52.096109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.108098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a0b86-a413-47b9-b186-52872db71612 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.217442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.218131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.218418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.218698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.219469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.221076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17eebb-689e-48c6-a0b7-9e44e05d6791 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.235313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1000/2001] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:52.236741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.248432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2ef96-648c-426a-8de9-c3a19e18b20f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.253371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.254309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.254583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.254856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] PATH_INFO: `/v3/projects/18942b14df804b998a3342ad2fe18ab6/users/b9cd204bb43041da82b2be7b9df878fe/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.259953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18942b14df804b998a3342ad2fe18ab6, user_id=b9cd204bb43041da82b2be7b9df878fe, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.262601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.279380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1002/2002] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:52 2022] PUT /identity/v3/projects/18942b14df804b998a3342ad2fe18ab6/users/b9cd204bb43041da82b2be7b9df878fe/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:52.288218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.288679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.289953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.318727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] MFA Rules not processed for user `b9cd204bb43041da82b2be7b9df878fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:52.319985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:52.397069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1001/2003] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:52.408191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.409909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:52.469869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c7cb9-8ca5-4b36-8cb1-e740a452d58f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.473576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.474287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.474568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.474844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.484052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.485484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c812ce-a630-4169-bf91-4a2bf44b82c8 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.495432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1003/2004] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:52.528069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.609966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939d891-9446-4686-a8fd-05db00ee929c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.616867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.617971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.618265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.618882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] PATH_INFO: `/v3/users/dcb7104515774edcb36ebd09850eb609` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.619813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb7104515774edcb36ebd09850eb609)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.621772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.647728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:52.680082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb7104515774edcb36ebd09850eb609'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:52.688716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb7104515774edcb36ebd09850eb609'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:52.701723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb7104515774edcb36ebd09850eb609'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:52.702318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb7104515774edcb36ebd09850eb609'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:52.709898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb7104515774edcb36ebd09850eb609'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:52.715951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1002/2005] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:52 2022] DELETE /identity/v3/users/dcb7104515774edcb36ebd09850eb609 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:52.720650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.720942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.721226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.739159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:52.740504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:52.803713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1004/2006] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:52.813531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:52.815065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:52.881477 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d88606-4b4c-4fd7-9bea-3d34233c0e66 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:52.886316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:52.886316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:52.886316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:52.886612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:52.896832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:52.898273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:52.909549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1003/2007] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:53.316696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:53.318945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:53.331510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ced9c5-1b59-474d-b7ff-e346723f3c66 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:53.337521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:53.338599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:53.338984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:53.339906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] PATH_INFO: `/v3/projects/38128058425e4bb3ac5e5f8b4690875e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:53.341006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38128058425e4bb3ac5e5f8b4690875e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:53.342981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:53.416277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38128058425e4bb3ac5e5f8b4690875e', 'request_id': 'req-1df92d09-97e6-453b-b881-54a8e7164b00'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.422513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38128058425e4bb3ac5e5f8b4690875e', 'request_id': 'req-1df92d09-97e6-453b-b881-54a8e7164b00'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.431499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38128058425e4bb3ac5e5f8b4690875e', 'request_id': 'req-1df92d09-97e6-453b-b881-54a8e7164b00'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.450377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:53.454573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] The token cache is being invalidate because project 38128058425e4bb3ac5e5f8b4690875e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:53.455308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.456302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.458858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1005/2008] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:53 2022] DELETE /identity/v3/projects/38128058425e4bb3ac5e5f8b4690875e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:53.463112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:53.473077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0acc8-71db-4f0e-9cf1-1bb3f7da6898 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:53.534396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:53.535271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:53.535587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:53.535863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] PATH_INFO: `/v3/users/b9cd204bb43041da82b2be7b9df878fe` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:53.537113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9cd204bb43041da82b2be7b9df878fe)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:53.539263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:53.584563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:53.587413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9cd204bb43041da82b2be7b9df878fe'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.595860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9cd204bb43041da82b2be7b9df878fe'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.609570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9cd204bb43041da82b2be7b9df878fe'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.610235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9cd204bb43041da82b2be7b9df878fe'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.619776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3a6f2bd2-9aed-4ddc-b207-92eabfdf8df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9cd204bb43041da82b2be7b9df878fe'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:53.626155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1004/2009] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:53 2022] DELETE /identity/v3/users/b9cd204bb43041da82b2be7b9df878fe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:54.143757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.145156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:54.160165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df92d09-97e6-453b-b881-54a8e7164b00 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.273467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.274436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.274781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.274989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] PATH_INFO: `/v3/projects/18942b14df804b998a3342ad2fe18ab6` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.276281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18942b14df804b998a3342ad2fe18ab6)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.278265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.330013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18942b14df804b998a3342ad2fe18ab6', 'request_id': 'req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:54.337075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18942b14df804b998a3342ad2fe18ab6', 'request_id': 'req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:54.344277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18942b14df804b998a3342ad2fe18ab6', 'request_id': 'req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:54.356164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] The token cache is being invalidate because project 18942b14df804b998a3342ad2fe18ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:54.356460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:54.357123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-286b9f2d-f404-4109-8ec6-e79b0b0a7fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:54.358698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1006/2010] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:54 2022] DELETE /identity/v3/projects/18942b14df804b998a3342ad2fe18ab6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:54.363646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.363987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.364277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.382756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:54.384081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:54.441887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1005/2011] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:54.449984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.450272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.450548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.471739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:54.473028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:54.481337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1007/2012] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:54.485481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.496028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de59c1-bc13-4984-aeca-8998a87a704b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.501033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.501704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.501981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.502251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.502970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:54.503273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.504926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.518014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1006/2013] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:54.522393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.532081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ccdeb-7c74-4c81-9b59-26c1e8c10c7a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.536785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.537455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.537751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.538014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.538720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.540947 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.581683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1008/2014] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:54 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:54.583603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.606087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cef36-e235-4b9c-bc3f-339202aa0e50 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.741318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.742227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.742772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.743230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.744136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.746365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.798813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1007/2015] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:54 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:54.804925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.830042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afd704-9dce-4b4d-ac7e-660b322647ca admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.836666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.837822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.838580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.838999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.840171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.843253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.861009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.863618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1009/2016] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:54.888535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96872-f7b0-4323-87ce-02137f818d8d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.898596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.899449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.899734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.900012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.900744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.902340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026d9b3-6311-4c06-bd98-4b6882b45dcd admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:54.924936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1008/2017] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:54.926591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:54.946173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c68c5-d981-4a29-9dce-8e5f8ed03b36 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:54.952552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:54.953718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:54.954068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:54.955498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] PATH_INFO: `/v3/projects/deabb0192500491395b52e6679820f3d/users/6d9ce4d1c8794469b50437d218193800/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:54.977664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deabb0192500491395b52e6679820f3d, user_id=6d9ce4d1c8794469b50437d218193800, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:54.977664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.003323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1010/2018] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:54 2022] PUT /identity/v3/projects/deabb0192500491395b52e6679820f3d/users/6d9ce4d1c8794469b50437d218193800/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:55.010111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.010598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.010971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.075830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] MFA Rules not processed for user `6d9ce4d1c8794469b50437d218193800`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:55.077235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:55.101028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:55.211691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1009/2019] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:55.212907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.260395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:55.302601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b033-acd4-4704-8664-226a2bb7ea54 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.308027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.308727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.309008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.309563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.310116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.312048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.340697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1011/2020] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:55 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:55.344511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.357127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd7e3-9e9b-4251-a59d-36f79c40b364 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.436073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.436988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.438034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.438331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.439101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.441118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.464181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1010/2021] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:55 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:55.468936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.480729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a9e66-2ac9-4796-a86c-304d74ce7162 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.485847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.486952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.486952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.487280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.487987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.489694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.501880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1012/2022] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:55.506599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.517403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c6b1b-e2df-4321-affb-1041f92c5f9e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.525427 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.526864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.527219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.527549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] PATH_INFO: `/v3/projects/2ca595eb21fa433f8e27a5d74de92241/users/b660e25e96bf406395ccc380a845fa15/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.546175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca595eb21fa433f8e27a5d74de92241, user_id=b660e25e96bf406395ccc380a845fa15, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.548934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.568613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1011/2023] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:55 2022] PUT /identity/v3/projects/2ca595eb21fa433f8e27a5d74de92241/users/b660e25e96bf406395ccc380a845fa15/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:55.572831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.583396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96c0c6-a337-4955-b8d7-3ec3f6405cf6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.643437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.644032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.644312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.644583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.645307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.647349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df777c7c-ea5c-4577-beac-4f6bbb773440 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.661192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1013/2024] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:55.663094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.674266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb34601-1d58-47a8-b98c-9df7970a6736 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.679994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.680893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.681170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.681437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] PATH_INFO: `/v3/projects/2ca595eb21fa433f8e27a5d74de92241/users/b660e25e96bf406395ccc380a845fa15/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.683440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca595eb21fa433f8e27a5d74de92241, user_id=b660e25e96bf406395ccc380a845fa15, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.686068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.697379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1012/2025] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:55 2022] PUT /identity/v3/projects/2ca595eb21fa433f8e27a5d74de92241/users/b660e25e96bf406395ccc380a845fa15/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:55.707752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.708045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.708501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.737066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] MFA Rules not processed for user `b660e25e96bf406395ccc380a845fa15`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:55.738403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:55.826222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1014/2026] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:55.838665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:55.840361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:55.918963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97d3e-3c0c-48de-93f3-2d7a8a0a5ba1 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:55.923373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:55.924147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:55.924529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:55.925130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:55.941690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:55.944660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:55.957811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1013/2027] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:56.039167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:56.050348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edaf6f-1ade-4d8e-b2c5-b852286b8f82 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:56.054472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:56.055199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:56.055589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:56.055931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:56.075952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:56.115631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:56.117069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:56.126096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1015/2028] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:56.430920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:56.437702 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:56.451372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beab7e6-fc80-4481-9993-e298a1ac4f21 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:56.462594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:56.463464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:56.463817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:56.464156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:56.466244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:56.480241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:56.482791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:56.580738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d185-4f26-4dff-8211-d2a6631accd9 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:56.591942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:56.594474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e9498-0679-4aaf-b0e9-fe500a8588cd service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:56.595203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:56.596311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:56.596467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:56.596772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] PATH_INFO: `/v3/users/6d9ce4d1c8794469b50437d218193800` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:56.601360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9ce4d1c8794469b50437d218193800)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:56.603315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:56.616950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1014/2029] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:56.676043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9ce4d1c8794469b50437d218193800'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:56.685607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9ce4d1c8794469b50437d218193800'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:56.686574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9ce4d1c8794469b50437d218193800'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:56.706643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9ce4d1c8794469b50437d218193800'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:56.717424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9ce4d1c8794469b50437d218193800'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:56.724666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1016/2030] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:56 2022] DELETE /identity/v3/users/6d9ce4d1c8794469b50437d218193800 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:56.729415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:56.729875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:56.730296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:56.785950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:56.790245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:56.890404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1015/2031] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:56.908811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:56.910707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:56.968890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:56.990204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a415484-fb6e-4712-9163-d37bcfa75dd0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:56.994479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:56.995180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:56.995536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:56.995997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:57.013060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:57.016107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:57.036975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1017/2032] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:57.595001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:57.596508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:57.615339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190dca9-9272-43cb-bf66-4416c5e901c0 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:57.638659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:57.639788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:57.640223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:57.640693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] PATH_INFO: `/v3/projects/deabb0192500491395b52e6679820f3d` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:57.644597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=deabb0192500491395b52e6679820f3d)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:57.646606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:57.750544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:57.752114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:57.754659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deabb0192500491395b52e6679820f3d', 'request_id': 'req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.766817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deabb0192500491395b52e6679820f3d', 'request_id': 'req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.776306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303afcd6-d154-40c8-897d-c27f1b39a4cf service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:57.777235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deabb0192500491395b52e6679820f3d', 'request_id': 'req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.795844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] The token cache is being invalidate because project deabb0192500491395b52e6679820f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:57.796454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.797766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.799945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1016/2033] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:57 2022] DELETE /identity/v3/projects/deabb0192500491395b52e6679820f3d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:57.804048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:57.805961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:57.852095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:57.853246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:57.853653 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:57.854073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] PATH_INFO: `/v3/users/d77ebd7fa57940edaab782c20e031f6a` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:57.855471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77ebd7fa57940edaab782c20e031f6a)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:57.856454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:57.858394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:57.937056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bc0e5-2fab-4f0b-9c84-1d520436bf0e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:57.947014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:57.947942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:57.948232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:57.948509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] PATH_INFO: `/v3/users/b660e25e96bf406395ccc380a845fa15` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:57.949916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b660e25e96bf406395ccc380a845fa15)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:57.952264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:57.965626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77ebd7fa57940edaab782c20e031f6a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.980808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77ebd7fa57940edaab782c20e031f6a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:57.981552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77ebd7fa57940edaab782c20e031f6a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.013465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77ebd7fa57940edaab782c20e031f6a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.027363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-de14cb4c-2994-4cb1-affc-0b1411195306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77ebd7fa57940edaab782c20e031f6a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.038835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1018/2034] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:57 2022] DELETE /identity/v3/users/d77ebd7fa57940edaab782c20e031f6a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:58.044727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:58.045014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:58.045457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:58.055574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b660e25e96bf406395ccc380a845fa15'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.064956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b660e25e96bf406395ccc380a845fa15'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.072809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:58.074422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:58.079818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b660e25e96bf406395ccc380a845fa15'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.080776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b660e25e96bf406395ccc380a845fa15'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.095035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b660e25e96bf406395ccc380a845fa15'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.107387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1017/2035] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:57 2022] DELETE /identity/v3/users/b660e25e96bf406395ccc380a845fa15 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:58.188683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1019/2036] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:58.202021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:58.203575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:58.333341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac91b-affc-410f-8cde-e2a5e3f0dc90 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:58.349535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:58.351123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:58.351683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:58.352069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:58.501689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:58.503225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8501c-fb5b-42a5-8f4b-640d5b7335c9 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:58.527842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1018/2037] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:36:58.600359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:58.602124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:58.616833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:58.631136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ec096-22d1-44e3-98df-6744f962c6aa None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:58.639582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:58.640654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:58.641225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:58.642045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] PATH_INFO: `/v3/projects/2ca595eb21fa433f8e27a5d74de92241` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:58.644172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca595eb21fa433f8e27a5d74de92241)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:58.647031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:58.717235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca595eb21fa433f8e27a5d74de92241', 'request_id': 'req-341ab844-89c9-4ca7-abd6-56df059b9b02'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.726627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca595eb21fa433f8e27a5d74de92241', 'request_id': 'req-341ab844-89c9-4ca7-abd6-56df059b9b02'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.752463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca595eb21fa433f8e27a5d74de92241', 'request_id': 'req-341ab844-89c9-4ca7-abd6-56df059b9b02'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.780508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] The token cache is being invalidate because project 2ca595eb21fa433f8e27a5d74de92241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:58.781063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.782991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-341ab844-89c9-4ca7-abd6-56df059b9b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:58.788097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1020/2038] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:58 2022] DELETE /identity/v3/projects/2ca595eb21fa433f8e27a5d74de92241 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:58.798765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:58.799765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:58.800146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:58.808893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:58.826617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:58.829470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:58.957770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1019/2039] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:36:58.968614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:58.968916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:58.969260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.002930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:59.004118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.022790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1021/2040] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:59.027958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.043605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbe585-b998-417e-a37b-fe214a6ded3c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.054304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.055120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.055524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.055880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.056885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:36:59.057265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.059022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.070939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.072671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1020/2041] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:36:59.075907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.077347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.092345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912a4ce-be59-4f4b-bf04-94b29b325a6b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.093031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cc708-dc24-49eb-b951-fca81c0c6500 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.100071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.101282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.101404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.101834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.102725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.104855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.107684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.108552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.109984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.110331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] PATH_INFO: `/v3/projects/0fd6036befbe4dfd91e017249d5b0b57` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.115747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd6036befbe4dfd91e017249d5b0b57)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.122199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.154309 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1021/2042] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:36:59.155698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.191695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd6036befbe4dfd91e017249d5b0b57', 'request_id': 'req-a9bb2092-688e-4199-a9dd-b8111d934fe2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.201661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd6036befbe4dfd91e017249d5b0b57', 'request_id': 'req-a9bb2092-688e-4199-a9dd-b8111d934fe2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.211447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd6036befbe4dfd91e017249d5b0b57', 'request_id': 'req-a9bb2092-688e-4199-a9dd-b8111d934fe2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.229707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] The token cache is being invalidate because project 0fd6036befbe4dfd91e017249d5b0b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:36:59.230081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.230864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.232628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1022/2043] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:36:59 2022] DELETE /identity/v3/projects/0fd6036befbe4dfd91e017249d5b0b57 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:59.248701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54d560-909d-4b62-9de2-03817a175453 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.324294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.325028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.325316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.325736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.326565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.328889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.367532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1022/2044] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:59.367625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.369354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.385915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb2092-688e-4199-a9dd-b8111d934fe2 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.392134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.393147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.393369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.393716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.394247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.394710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.396881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.397103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.407940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:59.409162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1023/2045] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:59.413819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.421873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6b58b-5384-4db5-adec-baed39f2695a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.426724 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d370-6579-47fa-981a-33992d0e3ccf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.431916 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.432894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.433180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.433452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] PATH_INFO: `/v3/users/f3312ef8ca6f455e87fbbbfd0b9a9664` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.434792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3312ef8ca6f455e87fbbbfd0b9a9664)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.437463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.445232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.446031 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.446325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.446588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.447358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.448985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.461642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1024/2046] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:36:59.465783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.478884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e51eb5-0985-4ffc-9c44-ad4afb663970 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.484704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.485517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.485801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.486091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] PATH_INFO: `/v3/projects/10cd59a54abb4f6f99922aabbafe8c62/users/87f4a0b813264a04a058fc54fd8fd3ff/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.497998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3312ef8ca6f455e87fbbbfd0b9a9664'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.503023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10cd59a54abb4f6f99922aabbafe8c62, user_id=87f4a0b813264a04a058fc54fd8fd3ff, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.506186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3312ef8ca6f455e87fbbbfd0b9a9664'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.506712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3be9f-84f4-4789-8b4e-092f242b8374 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.526504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1025/2047] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:59 2022] PUT /identity/v3/projects/10cd59a54abb4f6f99922aabbafe8c62/users/87f4a0b813264a04a058fc54fd8fd3ff/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:36:59.530245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3312ef8ca6f455e87fbbbfd0b9a9664'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.530998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3312ef8ca6f455e87fbbbfd0b9a9664'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.538376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.538656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.538935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.540164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-88be6ae4-3a24-4921-ac68-3f56ee3e0a62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3312ef8ca6f455e87fbbbfd0b9a9664'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:36:59.546822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1023/2048] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:36:59 2022] DELETE /identity/v3/users/f3312ef8ca6f455e87fbbbfd0b9a9664 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:59.550837 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.551120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.551419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.568288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] MFA Rules not processed for user `87f4a0b813264a04a058fc54fd8fd3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:59.569506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.573683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:59.580052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:36:59.581396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.657673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1026/2049] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:59.662411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1024/2050] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:36:59.663576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.665378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.671912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.673226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:36:59.680285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a5f0-a70c-4173-8e71-d5c6641de925 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.685066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.685750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.686108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.686389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.687198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.689268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.721110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1027/2051] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:59.722188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.735008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb1e2e-13db-49d0-bbf2-8bdd7899c1e6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.764367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:36:59.786128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d465a2-0c67-40b7-925c-d23c57660935 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.787165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.787687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.788101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.788436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.792731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.793495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.793853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.794192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.794995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.798045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.798985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.800508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.812989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1025/2052] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:36:59.828063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1028/2053] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:36:59 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:36:59.830634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.841683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b84fdc-0609-418a-aef7-6cabecdbecc7 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.846517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.847182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.847498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.847770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.848487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.850199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.860528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1026/2054] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:36:59.865430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.880851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa387818-8a3e-48f5-920f-b5875c571e6d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:36:59.885926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:36:59.886836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:36:59.887119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:36:59.887437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] PATH_INFO: `/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411/users/edbfc7ed16f14f0e9f1ed7894085499e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:36:59.904296 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c572324cc5a4a3a9fbc0a3d0bfb9411, user_id=edbfc7ed16f14f0e9f1ed7894085499e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:36:59.907001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:36:59.919698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1029/2055] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:36:59 2022] PUT /identity/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411/users/edbfc7ed16f14f0e9f1ed7894085499e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:36:59.923858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:36:59.938405 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8bee1-6092-404c-b6b9-7dde6fc64371 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:00.003094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:00.003926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.004390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.004685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.006020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:00.007847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4e465-a561-4d6e-bb26-24b737e14cfe admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:00.018313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1027/2056] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:00.022726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:00.032863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968b0bb-d066-4084-ac1d-bb78e8b395a5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:00.038295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:00.040125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.040487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.040835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] PATH_INFO: `/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411/users/edbfc7ed16f14f0e9f1ed7894085499e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.042972 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c572324cc5a4a3a9fbc0a3d0bfb9411, user_id=edbfc7ed16f14f0e9f1ed7894085499e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:00.045758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:00.061890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1030/2057] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:00 2022] PUT /identity/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411/users/edbfc7ed16f14f0e9f1ed7894085499e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:00.073255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.073528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.073804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.101603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] MFA Rules not processed for user `edbfc7ed16f14f0e9f1ed7894085499e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:00.103051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:00.194616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1028/2058] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:00.203513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:00.205466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:00.212776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:00.214862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:00.273435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:00.275178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:00.292176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5db55a-ee69-44ad-8770-63c0147243b4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:00.293423 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35a41a-f86f-4555-a463-c7604e3da688 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:00.296106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:00.296785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.297182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.297468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.298099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:00.298956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.299241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.299560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] PATH_INFO: `/v3/projects/575fd53c944e40f4a0ba834c28c73d52` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.300936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575fd53c944e40f4a0ba834c28c73d52)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:00.303891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:00.358992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575fd53c944e40f4a0ba834c28c73d52', 'request_id': 'req-245ff160-570b-4b21-b6df-87ce3b7b0ee5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.363994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:00.366917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:00.368022 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575fd53c944e40f4a0ba834c28c73d52', 'request_id': 'req-245ff160-570b-4b21-b6df-87ce3b7b0ee5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.374065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575fd53c944e40f4a0ba834c28c73d52', 'request_id': 'req-245ff160-570b-4b21-b6df-87ce3b7b0ee5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.376971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1031/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:00.386708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] The token cache is being invalidate because project 575fd53c944e40f4a0ba834c28c73d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:00.386995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.387716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.389447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1029/2060] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:00 2022] DELETE /identity/v3/projects/575fd53c944e40f4a0ba834c28c73d52 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:00.393439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:00.406126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2630919b-5473-44ab-be38-04973cf4be20 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:00.473614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:00.474605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:00.474912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:00.475236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] PATH_INFO: `/v3/users/ee925e3bd2654f4e97022232f68985b5` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:00.476432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee925e3bd2654f4e97022232f68985b5)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:00.478604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:00.528605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee925e3bd2654f4e97022232f68985b5'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.537606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee925e3bd2654f4e97022232f68985b5'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.538511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee925e3bd2654f4e97022232f68985b5'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.558105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee925e3bd2654f4e97022232f68985b5'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.569096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee925e3bd2654f4e97022232f68985b5'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:00.579126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1032/2061] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:00 2022] DELETE /identity/v3/users/ee925e3bd2654f4e97022232f68985b5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:00.926565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:00.928428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:00.950084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ff160-570b-4b21-b6df-87ce3b7b0ee5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.007780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:01.008648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.008931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.009206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] PATH_INFO: `/v3/projects/a0217eb4d9b947edac57aa034adeb4fa` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.010446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0217eb4d9b947edac57aa034adeb4fa)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:01.012921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:01.064518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0217eb4d9b947edac57aa034adeb4fa', 'request_id': 'req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.072353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0217eb4d9b947edac57aa034adeb4fa', 'request_id': 'req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.079627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0217eb4d9b947edac57aa034adeb4fa', 'request_id': 'req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.091342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] The token cache is being invalidate because project a0217eb4d9b947edac57aa034adeb4fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:01.091637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.092283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.094162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1030/2062] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:00 2022] DELETE /identity/v3/projects/a0217eb4d9b947edac57aa034adeb4fa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:01.097744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:01.100424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:01.113047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1c6e4-5413-4ca0-a760-2145c15c5adf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.118166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:01.175814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:01.176707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.177028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.177304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] PATH_INFO: `/v3/users/9a22427801ca4b7d966b030e66c7ccb8` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.178530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a22427801ca4b7d966b030e66c7ccb8)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:01.180489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:01.229846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a22427801ca4b7d966b030e66c7ccb8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.237830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a22427801ca4b7d966b030e66c7ccb8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.238446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a22427801ca4b7d966b030e66c7ccb8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.252070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a22427801ca4b7d966b030e66c7ccb8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.258879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3c0f5f18-4360-443d-a85b-d6fbfea76fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a22427801ca4b7d966b030e66c7ccb8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.264924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1033/2063] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:01 2022] DELETE /identity/v3/users/9a22427801ca4b7d966b030e66c7ccb8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:01.543810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:01.545305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:01.558864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acdb02-c345-4c2f-83e0-fa1aee6fc61e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.562563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:01.621789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:01.622796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.623079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.623384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] PATH_INFO: `/v3/projects/96b458659b6940f2b85076517a807045` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.624615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b458659b6940f2b85076517a807045)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:01.627150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:01.676399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b458659b6940f2b85076517a807045', 'request_id': 'req-48fc5cbb-3493-4a47-abab-25823b4f34c4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.685810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b458659b6940f2b85076517a807045', 'request_id': 'req-48fc5cbb-3493-4a47-abab-25823b4f34c4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.691867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b458659b6940f2b85076517a807045', 'request_id': 'req-48fc5cbb-3493-4a47-abab-25823b4f34c4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.705021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] The token cache is being invalidate because project 96b458659b6940f2b85076517a807045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:01.705316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.706013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-48fc5cbb-3493-4a47-abab-25823b4f34c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:01.708052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1031/2064] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:01 2022] DELETE /identity/v3/projects/96b458659b6940f2b85076517a807045 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:01.715306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.715609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.715891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.734151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:01.735691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:01.795623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1034/2065] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:01.804258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.804622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.804985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.829809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:01.831028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:01.850349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1032/2066] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:01.854531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:01.873627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72a018-33ba-46e9-a27a-9014eba66a56 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.878806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:01.879657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.879971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.880747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.881473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:01.881823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:01.883444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:01.904554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1035/2067] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:01.905767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:01.916906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec39aa-7a11-4a86-b269-156f037d69dc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.921660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:01.922340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:01.922599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:01.922875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:01.923602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:01.925506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:01.953348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1033/2068] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:01 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:01.957525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:01.968119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f1ff4-d15a-4d57-b520-d17a69d3b059 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:01.979473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:02.032079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.032762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.033054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.033331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.034022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.035994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.060816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1036/2069] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:01 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:02.064924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.076265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb83d8-1614-4e76-819f-415c5d9a048d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.081196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.081855 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.082139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.082412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.083213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.085441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.096697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1034/2070] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:02.102155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.112066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a517bdf-fc3a-4d3a-b275-d061416d795b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.117413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.118146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.118471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.118776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.119625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.121357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd946e4-4c49-4970-9da5-83a75340d827 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.142423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1037/2071] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:02.147326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.159322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82cbf8-7e37-443d-9d4e-849c5b437392 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.166402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.167722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.168213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.169212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] PATH_INFO: `/v3/projects/3ec1719098af46bdb0e8bda4fa96bee0/users/e44c424bbee64e80877d25112de15cf5/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.195406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec1719098af46bdb0e8bda4fa96bee0, user_id=e44c424bbee64e80877d25112de15cf5, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.199312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.226534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1035/2072] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:02 2022] PUT /identity/v3/projects/3ec1719098af46bdb0e8bda4fa96bee0/users/e44c424bbee64e80877d25112de15cf5/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:02.239800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.240310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.240450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.273186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] MFA Rules not processed for user `e44c424bbee64e80877d25112de15cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:02.276303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:02.363553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1038/2073] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:02.368537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.417169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:02.451799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5321d2f-2163-4b22-ac97-d149b343994b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.456625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.457320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.457605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.457883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.459481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.461393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.496132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1036/2074] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:02 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:02.497870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.511172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076c4b4-9c6c-4f8e-af91-192460741d4f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.571128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.571999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.572283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.572580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.573293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.575226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.596678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1039/2075] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:02 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:02.601033 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.611400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bafe54-3512-4571-b6c9-88070769e9ab admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.616717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.617377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.617664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.617941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.618672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.620299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.633352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1037/2076] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:02.637497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.647378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9816f7-49a6-41ba-9ed4-e008a861fd68 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.652233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.653107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.653386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.653660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] PATH_INFO: `/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15/users/16d029cd372f49d4a1077b9e31bda20e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.666304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:02.671523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b1e949f9cb4f97a35b9b0dd0a4aa15, user_id=16d029cd372f49d4a1077b9e31bda20e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.674641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.688176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1040/2077] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:02 2022] PUT /identity/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15/users/16d029cd372f49d4a1077b9e31bda20e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:02.692281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.702612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a0a02-fba9-4120-91d5-d0354b6589df admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.762595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.763314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.763646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.763929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.764664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.766299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca93cbf-a452-4e3e-acce-7516016d9148 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.777653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1038/2078] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:02.781769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.792404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b0d4e-098d-4ff6-9333-6c9c21fa1342 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:02.797021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:02.797922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.798205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.798479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] PATH_INFO: `/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15/users/16d029cd372f49d4a1077b9e31bda20e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.800504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b1e949f9cb4f97a35b9b0dd0a4aa15, user_id=16d029cd372f49d4a1077b9e31bda20e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:02.803393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:02.813441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1041/2079] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:02 2022] PUT /identity/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15/users/16d029cd372f49d4a1077b9e31bda20e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:02.824593 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:02.824887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:02.825274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:02.853774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] MFA Rules not processed for user `16d029cd372f49d4a1077b9e31bda20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:02.854998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:02.926669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1039/2080] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:02.935830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:02.937522 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:02.999049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b702400-f5d4-4e3c-9a6a-255051807291 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:03.002670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:03.003351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:03.003635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:03.003905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:03.013660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:03.015080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069a3d-18cb-4386-a9ec-5f3db44756a2 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:03.025099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1042/2081] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:03.764272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:03.798495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:03.836379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062755-6e47-453a-9f51-65c2a904d8b6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:03.841061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:03.841967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:03.842255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:03.842526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] PATH_INFO: `/v3/users/e44c424bbee64e80877d25112de15cf5` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:03.843761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44c424bbee64e80877d25112de15cf5)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:03.845692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:03.898762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44c424bbee64e80877d25112de15cf5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:03.907161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44c424bbee64e80877d25112de15cf5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:03.920280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44c424bbee64e80877d25112de15cf5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:03.920907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44c424bbee64e80877d25112de15cf5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:03.928001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44c424bbee64e80877d25112de15cf5'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:03.933809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1040/2082] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:03 2022] DELETE /identity/v3/users/e44c424bbee64e80877d25112de15cf5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:03.938289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:03.938795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:03.938890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:03.955687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:03.956837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:04.015370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1043/2083] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:04.026647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:04.028196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:04.086063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709424c6-dfad-4fef-b7c9-6e1892262b5c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:04.089623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:04.090264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:04.090545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:04.090813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:04.100264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:04.102018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:04.111251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1041/2084] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:04.385853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:04.387499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:04.407645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3e963-45ff-4e47-bf5e-dd428e68f391 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:04.412654 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:04.413631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:04.414016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:04.414464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] PATH_INFO: `/v3/projects/3ec1719098af46bdb0e8bda4fa96bee0` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:04.415758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec1719098af46bdb0e8bda4fa96bee0)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:04.417639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:04.466766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec1719098af46bdb0e8bda4fa96bee0', 'request_id': 'req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.472145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec1719098af46bdb0e8bda4fa96bee0', 'request_id': 'req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.479770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec1719098af46bdb0e8bda4fa96bee0', 'request_id': 'req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.491546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] The token cache is being invalidate because project 3ec1719098af46bdb0e8bda4fa96bee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:04.491828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.492539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.494098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1044/2085] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:04 2022] DELETE /identity/v3/projects/3ec1719098af46bdb0e8bda4fa96bee0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:04.498065 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:04.508501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21816f-4bf9-4a58-984b-43b7ee689a94 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:04.557468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:04.570306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:04.571179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:04.571486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:04.571755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] PATH_INFO: `/v3/users/16d029cd372f49d4a1077b9e31bda20e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:04.572949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d029cd372f49d4a1077b9e31bda20e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:04.574883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:04.622269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d029cd372f49d4a1077b9e31bda20e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.636089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d029cd372f49d4a1077b9e31bda20e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.649974 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d029cd372f49d4a1077b9e31bda20e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.650526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d029cd372f49d4a1077b9e31bda20e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.660890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d029cd372f49d4a1077b9e31bda20e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:04.671869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1042/2086] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:04 2022] DELETE /identity/v3/users/16d029cd372f49d4a1077b9e31bda20e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:04.736814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:04.738396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:04.744967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:04.817356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3f02d-4606-44c9-9cd5-1b4c66f5cc58 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:04.882062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:04.882717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:04.882996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:04.883267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:04.893186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:04.971357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:04.973550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cd597-3ed4-4ef2-9db5-dcaa83cb0b21 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:04.991926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1045/2087] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:05.004299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.006458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:05.098319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6017e-542e-494f-abf6-7e09ee95ae0b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.107892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.109060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.109339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.109733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] PATH_INFO: `/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.111028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b1e949f9cb4f97a35b9b0dd0a4aa15)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.113063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.163446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b1e949f9cb4f97a35b9b0dd0a4aa15', 'request_id': 'req-629fbe62-abf8-4bff-930a-ca318e5df1dc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:05.172031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b1e949f9cb4f97a35b9b0dd0a4aa15', 'request_id': 'req-629fbe62-abf8-4bff-930a-ca318e5df1dc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:05.178780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b1e949f9cb4f97a35b9b0dd0a4aa15', 'request_id': 'req-629fbe62-abf8-4bff-930a-ca318e5df1dc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:05.186648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:05.191619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] The token cache is being invalidate because project 01b1e949f9cb4f97a35b9b0dd0a4aa15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:05.192303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:05.193152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-629fbe62-abf8-4bff-930a-ca318e5df1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:05.195059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1043/2088] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:05 2022] DELETE /identity/v3/projects/01b1e949f9cb4f97a35b9b0dd0a4aa15 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:05.201029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.201715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.202224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.222199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:05.223402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:05.308409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1046/2089] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:05.310300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.311182 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.311400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.331895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:05.333093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:05.341552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1044/2090] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:05.345780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.356095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e98c9-33bc-41fc-b942-ae307cda34ab None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.366940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.367664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.367951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.368229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.368966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:05.369276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.370957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.373666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:05.384569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1047/2091] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:05.389081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.399973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121d5c8-d3d0-4337-a9fb-054fa825745c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.404756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.405411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.405690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.405965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.406661 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.408588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.438188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1045/2092] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:05 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:05.439390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.450817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cea6ff-0a8f-4d8b-89fe-0b96cf7751be admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.506827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.507525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.507812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.508089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.508774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.510691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.530065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1048/2093] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:05 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:05.534286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.545139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c809a4c-72a8-48e5-9fcb-b53d530603cb admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.549720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.550379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.550663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.550943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.551708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.553882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.564451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1046/2094] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:05.568468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.578575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd7cfd-f449-42c4-b846-43ceafb11dfb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.584272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.585001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.585302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.585608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.586363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.588091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340e9c7-b3d6-4454-b784-8f63009f7853 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.598720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1049/2095] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:05.603496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.613772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d366f-ac11-4bae-a6c2-e9d0a8bedc0f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.618764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.619686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.619972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.620250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] PATH_INFO: `/v3/projects/fc0fa26ef67f42cfa7fc4dfe94343900/users/5235468c0d91476bb35fa8ebf43ebf4b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.635676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0fa26ef67f42cfa7fc4dfe94343900, user_id=5235468c0d91476bb35fa8ebf43ebf4b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.638373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.651800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1047/2096] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:05 2022] PUT /identity/v3/projects/fc0fa26ef67f42cfa7fc4dfe94343900/users/5235468c0d91476bb35fa8ebf43ebf4b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:05.661871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.662165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.662446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.690750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] MFA Rules not processed for user `5235468c0d91476bb35fa8ebf43ebf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:05.692026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:05.757828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1050/2097] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:05.769224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.771002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:05.829464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c53fe-56e5-416f-96c4-4755df8849aa admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.833237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.834413 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.834705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.834985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.847721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.849217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd497e4f-59eb-426b-8d5f-73a27e312754 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.858739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1048/2098] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:05.878222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:05.948915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5639e-326b-452b-bf78-c29c36807c0a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:05.954006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:05.955029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:05.955378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:05.955731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] PATH_INFO: `/v3/users/5235468c0d91476bb35fa8ebf43ebf4b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:05.956960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5235468c0d91476bb35fa8ebf43ebf4b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:05.959419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:05.964421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:06.009902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5235468c0d91476bb35fa8ebf43ebf4b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.017700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5235468c0d91476bb35fa8ebf43ebf4b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.018752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5235468c0d91476bb35fa8ebf43ebf4b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.032776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5235468c0d91476bb35fa8ebf43ebf4b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.040993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5235468c0d91476bb35fa8ebf43ebf4b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.047167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1051/2099] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:05 2022] DELETE /identity/v3/users/5235468c0d91476bb35fa8ebf43ebf4b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:06.051606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.052209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.052512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.072297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:06.073524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:06.134161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1049/2100] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:06.144208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:06.146308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:06.208032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef829fc-e1c1-425e-bb88-fac2416a7c28 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:06.212939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:06.213591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.213867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.214141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.223885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:06.225494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250ce3d-9802-4bfe-a1d1-0eb324a0ac7c service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:06.236603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1052/2101] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:06.556525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:06.558331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:06.572018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fffffb-19ba-4610-839a-eafccdffbcfa None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:06.579219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:06.580316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.580622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.581007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] PATH_INFO: `/v3/projects/fc0fa26ef67f42cfa7fc4dfe94343900` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.582371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0fa26ef67f42cfa7fc4dfe94343900)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:06.584942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:06.640225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:06.654124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0fa26ef67f42cfa7fc4dfe94343900', 'request_id': 'req-eb706a1e-8434-4269-ab29-c5c7b2f87474'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.668137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0fa26ef67f42cfa7fc4dfe94343900', 'request_id': 'req-eb706a1e-8434-4269-ab29-c5c7b2f87474'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.674396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0fa26ef67f42cfa7fc4dfe94343900', 'request_id': 'req-eb706a1e-8434-4269-ab29-c5c7b2f87474'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.686511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] The token cache is being invalidate because project fc0fa26ef67f42cfa7fc4dfe94343900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:06.686869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.687660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-eb706a1e-8434-4269-ab29-c5c7b2f87474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:06.689415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1050/2102] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:06 2022] DELETE /identity/v3/projects/fc0fa26ef67f42cfa7fc4dfe94343900 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:06.694801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.695087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.695409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.714056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:06.715279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:06.783682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1053/2103] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:06.785923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.786214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.786496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.806084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:06.807318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:06.815257 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1051/2104] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:06.819571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:06.829042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41bd15-ea6c-47f5-a9f3-116518a36907 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:06.834136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:06.834795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.835072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.835421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.836183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:06.836484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:06.838103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:06.853992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1054/2105] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:06.855121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:06.870831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66e56c-97cd-4ec5-81c7-5dfad3eece62 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:06.875740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:06.876399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:06.876674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:06.876991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:06.877688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:06.879617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:06.906670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1052/2106] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:06 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:06.910907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:06.922338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8815976-3073-4710-8fec-39d09975e92e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:06.932765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:07.009368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.010730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.014026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.014482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.015442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.017770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.040744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1055/2107] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:06 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:07.045231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.059906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233b3aa-ea7d-4f42-9237-76f82337e015 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.065887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.066959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.067401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.067965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.068873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.070590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.084449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1053/2108] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:07.088938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.100543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d4be13-2b5f-41d9-8504-00b49eee3b20 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.111964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.115641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.116056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.116439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.117326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.119178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf43531-1bda-4f8d-a3a6-a34350718841 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.130857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1056/2109] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:07.135039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.145417 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279638a-7358-434b-a24f-5e5819505079 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.157353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.158341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.159198 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.159666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] PATH_INFO: `/v3/projects/7ac3a20cd2544d2991db498fb61cecb2/users/fffdab067a16497d8c06f9ad61d5af5a/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.179112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac3a20cd2544d2991db498fb61cecb2, user_id=fffdab067a16497d8c06f9ad61d5af5a, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.182232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.211679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1054/2110] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:07 2022] PUT /identity/v3/projects/7ac3a20cd2544d2991db498fb61cecb2/users/fffdab067a16497d8c06f9ad61d5af5a/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:07.223080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.223474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.223679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.279162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] MFA Rules not processed for user `fffdab067a16497d8c06f9ad61d5af5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:07.280266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:07.375536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1057/2111] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:07.383951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.385676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:07.443207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2afa-90ff-4a68-aa97-39f121f83a41 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.447513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.448186 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.448640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.448936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.459264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.460786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0350343-043f-4472-b004-18f4ca127013 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.470100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1055/2112] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:07.489151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.556419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238da99-afee-41e2-90d7-2ceb1b9489d1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.561080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.561979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.562257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.562530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] PATH_INFO: `/v3/users/fffdab067a16497d8c06f9ad61d5af5a` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.563806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffdab067a16497d8c06f9ad61d5af5a)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.565718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.588578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:07.617675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffdab067a16497d8c06f9ad61d5af5a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:07.625584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffdab067a16497d8c06f9ad61d5af5a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:07.626269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffdab067a16497d8c06f9ad61d5af5a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:07.638526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffdab067a16497d8c06f9ad61d5af5a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:07.645376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffdab067a16497d8c06f9ad61d5af5a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:07.652697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1058/2113] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:07 2022] DELETE /identity/v3/users/fffdab067a16497d8c06f9ad61d5af5a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:07.657370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.657737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.658087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.679678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:07.682109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:07.760139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1056/2114] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:07.768067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:07.770409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:07.829495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358426e2-1586-443a-b53c-d574e8f83317 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:07.833682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:07.834339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:07.834626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:07.834962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:07.844166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:07.845631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970036f-c742-434e-a106-d990f5c6267b service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:07.856683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1059/2115] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:08.136565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.139124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:08.147571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:08.154866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4948fb8-42c6-4a4b-b948-d35e6feccb94 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.165071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.166129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.166467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.166819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] PATH_INFO: `/v3/projects/7ac3a20cd2544d2991db498fb61cecb2` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.168238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac3a20cd2544d2991db498fb61cecb2)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.170827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.227968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac3a20cd2544d2991db498fb61cecb2', 'request_id': 'req-ab130689-b1c7-4624-b84f-16f889dec48b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:08.241674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac3a20cd2544d2991db498fb61cecb2', 'request_id': 'req-ab130689-b1c7-4624-b84f-16f889dec48b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:08.248241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac3a20cd2544d2991db498fb61cecb2', 'request_id': 'req-ab130689-b1c7-4624-b84f-16f889dec48b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:08.260510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] The token cache is being invalidate because project 7ac3a20cd2544d2991db498fb61cecb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:08.260881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:08.261558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ab130689-b1c7-4624-b84f-16f889dec48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:08.263315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1057/2116] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:08 2022] DELETE /identity/v3/projects/7ac3a20cd2544d2991db498fb61cecb2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:08.268109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.268908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.268908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.289941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:08.291164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:08.378467 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1060/2117] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:08.380902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.381409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.381687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.424725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:08.426028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:08.435376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1058/2118] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:08.441926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.450781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf592-2cb2-4f24-a4ee-b2fd24b9ea1b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.455496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.456161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.456448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.456726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.457722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:08.457867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.459791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.473050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1061/2119] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:08.477111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.486294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fc8c8-b511-47ae-920f-75b713a5652c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.493028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.493755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.494107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.494451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.495215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.497206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.528066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1059/2120] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:08.530558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.543343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5b0d3-efd8-4174-bfef-8182b0c845c7 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.575858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:08.610555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.611232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.611551 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.611822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.612511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.614995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.641132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1062/2121] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:08.642050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.658653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd400d3b-2943-4937-b97e-c3c64ffbc128 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.664606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.666695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.666964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.667247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.667995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.669643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.683338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1060/2122] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:08.684499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.696444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fcbf-51a6-4fa3-9661-d08219026ec8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.701835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.702608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.702918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.703200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.704074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.705752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277c6b7-4ebe-42c4-8b56-18bc8c64a9db admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.719459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1063/2123] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:08.722808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.733217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8415b47-9c6a-484b-914e-26c5c2f284af admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.738227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.739123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.739496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.739773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] PATH_INFO: `/v3/projects/6a6985e1b2d346debec8fde407132f5a/users/d0e9578e9ccd47b1b6a0afdc2f415891/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.759554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6985e1b2d346debec8fde407132f5a, user_id=d0e9578e9ccd47b1b6a0afdc2f415891, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.762211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:08.778784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1061/2124] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:08 2022] PUT /identity/v3/projects/6a6985e1b2d346debec8fde407132f5a/users/d0e9578e9ccd47b1b6a0afdc2f415891/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:08.789996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.790287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.790567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.818540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] MFA Rules not processed for user `d0e9578e9ccd47b1b6a0afdc2f415891`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:08.819806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:08.900381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1064/2125] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:08.904918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:08.978174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa7350-0077-4a4b-bb57-94326e95eec1 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:08.983159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:08.984262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:08.984545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:08.984833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:08.985517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:08.987490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.012979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1062/2126] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:08 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:09.017169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.028809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75e96c-52b3-4c5d-8d89-6df8b3f8c3b3 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.090413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.091108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.091429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.091717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.092456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.094452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.118478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1065/2127] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:09 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:09.119897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.131793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a2fc6-314b-457f-a2c3-0a5a0216406d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.137312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.138107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.138491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.138731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.139509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.141828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.143380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:09.154925 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1063/2128] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:09.158918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.169390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a429e5-8cb8-4fd0-9028-316dbbbe29c5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.174064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.174950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.175223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.175538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] PATH_INFO: `/v3/projects/078a38ec90974b9db2c46603ddec035d/users/356cf610541e4e1a95b340b880d3abc7/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.190636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=078a38ec90974b9db2c46603ddec035d, user_id=356cf610541e4e1a95b340b880d3abc7, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.194243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.208774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1066/2129] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:09 2022] PUT /identity/v3/projects/078a38ec90974b9db2c46603ddec035d/users/356cf610541e4e1a95b340b880d3abc7/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:09.212992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.223758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc9a7d-47ce-49e0-8cb0-2b327a34c59f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.282923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.283630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.283910 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.284180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.284893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.286671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb7bb6-ff67-4678-8eb8-626f0bb61608 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.296637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1064/2130] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:09.300895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.308831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:09.312816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183be41-c961-4d57-812b-be90f063c3b0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.319737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.320634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.320921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.321200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] PATH_INFO: `/v3/projects/078a38ec90974b9db2c46603ddec035d/users/356cf610541e4e1a95b340b880d3abc7/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.323173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=078a38ec90974b9db2c46603ddec035d, user_id=356cf610541e4e1a95b340b880d3abc7, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.325824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.336241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1067/2131] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:09 2022] PUT /identity/v3/projects/078a38ec90974b9db2c46603ddec035d/users/356cf610541e4e1a95b340b880d3abc7/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:09.347079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.347402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.347681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.375607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] MFA Rules not processed for user `356cf610541e4e1a95b340b880d3abc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:09.376811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:09.448787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1065/2132] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:09.459993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.461768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:09.521793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5fcf6-c957-40ec-b238-cd414d2f73ae admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:09.525537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:09.526129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:09.526416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:09.527049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:09.587589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:09.589038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88031439-06c2-4157-b058-ad33e9db373d service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:09.598599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1068/2133] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:09.960026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:09.961511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:10.031665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a6dc9-bbad-4355-9bc8-bf06e318e1b2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:10.036254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:10.037503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:10.037789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:10.038069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] PATH_INFO: `/v3/users/87f4a0b813264a04a058fc54fd8fd3ff` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:10.039624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f4a0b813264a04a058fc54fd8fd3ff)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:10.041571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:10.064707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:10.096279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f4a0b813264a04a058fc54fd8fd3ff'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.106055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f4a0b813264a04a058fc54fd8fd3ff'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.120626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f4a0b813264a04a058fc54fd8fd3ff'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.121734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f4a0b813264a04a058fc54fd8fd3ff'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.130673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f4a0b813264a04a058fc54fd8fd3ff'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.137437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1066/2134] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:09 2022] DELETE /identity/v3/users/87f4a0b813264a04a058fc54fd8fd3ff => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:10.141975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:10.142557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:10.142902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:10.165637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:10.167234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:10.237577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1069/2135] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:10.248108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:10.249628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:10.312379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdb5e-efaa-4cf5-b73f-4f2a7370e5ab admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:10.316289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:10.316935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:10.317213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:10.317484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:10.326871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:10.328338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:10.340046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1067/2136] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:10.628471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:10.631715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:10.645097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94eac05-2b89-4208-8366-8ae95795a016 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:10.650061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:10.651975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:10.651975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:10.651975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] PATH_INFO: `/v3/projects/10cd59a54abb4f6f99922aabbafe8c62` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:10.652870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=10cd59a54abb4f6f99922aabbafe8c62)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:10.654867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:10.702630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:10.710076 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10cd59a54abb4f6f99922aabbafe8c62', 'request_id': 'req-14dd8c33-ccae-4431-9b5a-91968e7c73fd'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.716496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10cd59a54abb4f6f99922aabbafe8c62', 'request_id': 'req-14dd8c33-ccae-4431-9b5a-91968e7c73fd'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.724163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10cd59a54abb4f6f99922aabbafe8c62', 'request_id': 'req-14dd8c33-ccae-4431-9b5a-91968e7c73fd'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.741590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] The token cache is being invalidate because project 10cd59a54abb4f6f99922aabbafe8c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:10.741880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.742578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.744363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1070/2137] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:10 2022] DELETE /identity/v3/projects/10cd59a54abb4f6f99922aabbafe8c62 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:10.748242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:10.759122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0910e-db43-445e-85e8-780c736b7f0c service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:10.830001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:10.830821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:10.831111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:10.831425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] PATH_INFO: `/v3/users/edbfc7ed16f14f0e9f1ed7894085499e` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:10.832623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbfc7ed16f14f0e9f1ed7894085499e)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:10.835212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:10.886578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:10.893156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbfc7ed16f14f0e9f1ed7894085499e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.904997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbfc7ed16f14f0e9f1ed7894085499e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.931684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbfc7ed16f14f0e9f1ed7894085499e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.932891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbfc7ed16f14f0e9f1ed7894085499e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.939892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b32070f1-5664-4479-a049-03559192fb89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbfc7ed16f14f0e9f1ed7894085499e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:10.946740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1068/2138] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:10 2022] DELETE /identity/v3/users/edbfc7ed16f14f0e9f1ed7894085499e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:11.247446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.249740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:11.265739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd8c33-ccae-4431-9b5a-91968e7c73fd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.355195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.356198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.356803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.357187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] PATH_INFO: `/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.359087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c572324cc5a4a3a9fbc0a3d0bfb9411)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.362608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.439216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c572324cc5a4a3a9fbc0a3d0bfb9411', 'request_id': 'req-b33c7773-a248-4986-bc4b-d3f55b083831'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:11.441652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c572324cc5a4a3a9fbc0a3d0bfb9411', 'request_id': 'req-b33c7773-a248-4986-bc4b-d3f55b083831'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:11.473248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c572324cc5a4a3a9fbc0a3d0bfb9411', 'request_id': 'req-b33c7773-a248-4986-bc4b-d3f55b083831'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:11.488888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] The token cache is being invalidate because project 1c572324cc5a4a3a9fbc0a3d0bfb9411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:11.489373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:11.491129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b33c7773-a248-4986-bc4b-d3f55b083831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:11.497777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1071/2139] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:11 2022] DELETE /identity/v3/projects/1c572324cc5a4a3a9fbc0a3d0bfb9411 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:11.511073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.511581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.511700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.537709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:11.539756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:11.611144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1069/2140] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:11.620087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916db747-11f8-457d-b164-ae2889424b12 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.620376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916db747-11f8-457d-b164-ae2889424b12 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.620655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916db747-11f8-457d-b164-ae2889424b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.641242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-916db747-11f8-457d-b164-ae2889424b12 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:11.642484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-916db747-11f8-457d-b164-ae2889424b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:11.653447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1072/2141] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:11.657929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.668504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3669-4131-4714-aac7-d761e7b76f9c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.673505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.674172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.674457 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.674731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.675503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:11.675814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.677478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.691908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1070/2142] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:11.698444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916db747-11f8-457d-b164-ae2889424b12 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.708907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916db747-11f8-457d-b164-ae2889424b12 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.714078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.714763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.715034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.715334 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.716039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.717957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.749337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1073/2143] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:11 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:11.750549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.766549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714d743-c473-443e-92c7-73ce03683af4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.837225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.838146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.838436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.838714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.839444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.841374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.868141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1071/2144] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:11 2022] POST /identity/v3/users => generated 280 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:11.874508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.892775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee7aa0-f752-4729-a676-db847fe99080 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.899016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.899732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.900018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.900296 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.901030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.902879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.915542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1074/2145] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:11.919096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:11.935275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a553a2-e9b2-43b2-a28f-d9b5141971d4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:11.947884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:11.950300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:11.950691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:11.951066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:11.953044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:11.958123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04240f53-9c75-417e-acd6-af1f8de86821 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:11.987970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1072/2146] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:11.993351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.003647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecd964-5f5f-488b-a123-9c535e9b7696 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.010425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.010774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.011056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.011366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] PATH_INFO: `/v3/projects/6e5efb2913864d3a9359f5d005f1f631/users/d76d0a4d5b2d4b74ac44841a54ac3832/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.030505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5efb2913864d3a9359f5d005f1f631, user_id=d76d0a4d5b2d4b74ac44841a54ac3832, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.033224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.038057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:12.076022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1075/2147] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:11 2022] PUT /identity/v3/projects/6e5efb2913864d3a9359f5d005f1f631/users/d76d0a4d5b2d4b74ac44841a54ac3832/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:12.079268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.079672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.080026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.112993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] MFA Rules not processed for user `d76d0a4d5b2d4b74ac44841a54ac3832`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:12.114179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.128987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:12.203844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.206107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.235460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1073/2148] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:12.281648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.283533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.369132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7d631-339c-406e-8dd4-540787d4cb40 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.373438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.374101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.374391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.374667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.384971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.386455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.397076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1074/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:37:12.398293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a53b-d775-4f70-8d1a-806451bdfd0e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.404958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.406242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.406697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.406990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] PATH_INFO: `/v3/users/d0e9578e9ccd47b1b6a0afdc2f415891` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.408450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e9578e9ccd47b1b6a0afdc2f415891)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.410537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.420425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.437111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe533ac-8167-46b9-9e08-903bf503355e service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.445619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.446637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.446637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.447056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] PATH_INFO: `/v3/users/d76d0a4d5b2d4b74ac44841a54ac3832` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.448481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76d0a4d5b2d4b74ac44841a54ac3832)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.452151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.503381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e9578e9ccd47b1b6a0afdc2f415891'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.514478 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e9578e9ccd47b1b6a0afdc2f415891'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.515556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e9578e9ccd47b1b6a0afdc2f415891'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.532428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e9578e9ccd47b1b6a0afdc2f415891'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.542214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-3e062773-cbe6-4612-a5ea-9207b2b69d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e9578e9ccd47b1b6a0afdc2f415891'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.542587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76d0a4d5b2d4b74ac44841a54ac3832'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.551093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1076/2150] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:12 2022] DELETE /identity/v3/users/d0e9578e9ccd47b1b6a0afdc2f415891 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:12.556232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76d0a4d5b2d4b74ac44841a54ac3832'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.557931 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.558534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.559546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.572839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76d0a4d5b2d4b74ac44841a54ac3832'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.573583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76d0a4d5b2d4b74ac44841a54ac3832'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.581793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:12.583229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.584857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a13ef88-859f-4c44-a25b-1341f215b228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76d0a4d5b2d4b74ac44841a54ac3832'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:12.592069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1075/2151] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:12 2022] DELETE /identity/v3/users/d76d0a4d5b2d4b74ac44841a54ac3832 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:12.596412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.596829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.597312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.617344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:12.618753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.651801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:12.679688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1077/2152] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:12.688708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.690172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.727670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1076/2153] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:12.737585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:12.744665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:12.765196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:12.793678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a61ae-f067-4559-8fac-961ad174edff None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.849882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce2c1-9c03-4fb1-920a-80f591b35ce1 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:12.854970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.856199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.856555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.856897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.862655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:12.863521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:12.863882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:12.864317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:12.868580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.870409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.875133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:12.876892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:12.884139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1077/2154] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:12.897704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1078/2155] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:13.330484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:13.332669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:13.358339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110551c-6eb1-4eab-bf7c-4785eb5b844d service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:13.373764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:13.375123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:13.375267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:13.375973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] PATH_INFO: `/v3/projects/6a6985e1b2d346debec8fde407132f5a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:13.378822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6985e1b2d346debec8fde407132f5a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:13.386847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:13.510316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6985e1b2d346debec8fde407132f5a', 'request_id': 'req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.531835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6985e1b2d346debec8fde407132f5a', 'request_id': 'req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.551696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6985e1b2d346debec8fde407132f5a', 'request_id': 'req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.572325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] The token cache is being invalidate because project 6a6985e1b2d346debec8fde407132f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:13.572701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.574249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.575129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1078/2156] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:13 2022] DELETE /identity/v3/projects/6a6985e1b2d346debec8fde407132f5a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:13.579234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:13.591810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6281529-2bf1-4ccf-8eab-53020c23d3f2 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:13.672630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:13.673865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:13.674155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:13.674434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] PATH_INFO: `/v3/users/356cf610541e4e1a95b340b880d3abc7` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:13.676074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=356cf610541e4e1a95b340b880d3abc7)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:13.679165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:13.722880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:13.726167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:13.756851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd93ff2-f9f0-43d3-bda5-28c9f4f0d3e4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:13.761642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356cf610541e4e1a95b340b880d3abc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.772274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:13.773259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:13.774498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:13.774841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] PATH_INFO: `/v3/projects/6e5efb2913864d3a9359f5d005f1f631` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:13.776152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356cf610541e4e1a95b340b880d3abc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.776861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356cf610541e4e1a95b340b880d3abc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.779877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5efb2913864d3a9359f5d005f1f631)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:13.781880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:13.800569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356cf610541e4e1a95b340b880d3abc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.847822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:13.861679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-135edebc-9768-4586-b67d-e4bf8dd70968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356cf610541e4e1a95b340b880d3abc7'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.877005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1079/2157] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:13 2022] DELETE /identity/v3/users/356cf610541e4e1a95b340b880d3abc7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:13.896709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5efb2913864d3a9359f5d005f1f631', 'request_id': 'req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.898326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:13.913004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5efb2913864d3a9359f5d005f1f631', 'request_id': 'req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.923948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5efb2913864d3a9359f5d005f1f631', 'request_id': 'req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.943996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] The token cache is being invalidate because project 6e5efb2913864d3a9359f5d005f1f631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:13.946707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.947633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9a7b0a66-6de5-4330-bbd4-bb242f6ba622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:13.950104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1079/2158] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:13 2022] DELETE /identity/v3/projects/6e5efb2913864d3a9359f5d005f1f631 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:13.955859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:13.955859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:13.955859 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:13.987819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:13.990649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.118574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1080/2159] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:14.126458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.126745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.127031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.150090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:14.151364 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.163841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1080/2160] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:14.165270 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.176645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841334c4-a6f6-4a5d-8393-e44ae84a4fda None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.183691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.184642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.185355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.185793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.186680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:14.187136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.189847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.212010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1081/2161] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:14.214177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.226233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87b85-48b2-4015-8b28-f08c8796a82f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.232277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.233253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.233621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.234368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.235145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.237062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.270324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1081/2162] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:14.274951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.285513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdcfe4-4664-4c44-a530-d77662b5692d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.371823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.372242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.372677 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.373060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.373889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.376165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.414952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1082/2163] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:14.417040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.431751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d17e40-5771-45d2-b463-b783a32cbab7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.447480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.450096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.455403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.456128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.456411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.456687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.457418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.461476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.463713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbeafb-203c-46f4-98c9-906ef9ed80fd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.469395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.470903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.471520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.472338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] PATH_INFO: `/v3/projects/078a38ec90974b9db2c46603ddec035d` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.476922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] RBAC: Authorizing `identity:delete_project(project_id=078a38ec90974b9db2c46603ddec035d)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.480657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.497404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1082/2164] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:14.500873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.513081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3219-6a87-4220-812f-14ebead6b6e9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.518125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.518817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.519100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.519437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.520249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.522358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.535323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1083/2165] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:14.538663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.550905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff77d3-65fb-4686-9809-3cbe76283902 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.557603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078a38ec90974b9db2c46603ddec035d', 'request_id': 'req-af20d875-d135-42aa-87a0-d05f538f466b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:14.564878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078a38ec90974b9db2c46603ddec035d', 'request_id': 'req-af20d875-d135-42aa-87a0-d05f538f466b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:14.574245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078a38ec90974b9db2c46603ddec035d', 'request_id': 'req-af20d875-d135-42aa-87a0-d05f538f466b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:14.576025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:14.591157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] The token cache is being invalidate because project 078a38ec90974b9db2c46603ddec035d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:14.591790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:14.593004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-af20d875-d135-42aa-87a0-d05f538f466b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:14.598505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1083/2166] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:14 2022] DELETE /identity/v3/projects/078a38ec90974b9db2c46603ddec035d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:14.600647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30865bf4-9eaf-47ce-853e-80ffeb2130b2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.601150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30865bf4-9eaf-47ce-853e-80ffeb2130b2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.601635 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30865bf4-9eaf-47ce-853e-80ffeb2130b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.621258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-30865bf4-9eaf-47ce-853e-80ffeb2130b2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:14.622465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-30865bf4-9eaf-47ce-853e-80ffeb2130b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.624456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.625434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.625673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.626561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] PATH_INFO: `/v3/projects/53edaab7a665412c954dddcfab7ba106/users/b6251c77d2ee437791dcc35597fba5f1/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.632737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1084/2167] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:14.640923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.640923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.641194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.643438 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53edaab7a665412c954dddcfab7ba106, user_id=b6251c77d2ee437791dcc35597fba5f1, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.646181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c44fa-f0d1-4239-a16b-373e8f4d1f81 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.659654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1084/2168] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:14 2022] PUT /identity/v3/projects/53edaab7a665412c954dddcfab7ba106/users/b6251c77d2ee437791dcc35597fba5f1/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:14.659990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:14.661790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.671737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.672053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.672337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.700097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] MFA Rules not processed for user `b6251c77d2ee437791dcc35597fba5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:14.701495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.723326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1085/2169] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:14.728341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.737658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6dd28-c7b3-4667-ae3a-2dbcc1055d5b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.743311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.743934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.744219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.744493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.745449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:14.745575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.747156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.766096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1086/2170] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:37:14.766096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.776850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec2b58-7e38-498c-9b08-c338111c1f43 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.782855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.783632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.783991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.788230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.788406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1085/2171] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:14.788529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.788529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.789943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.791072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:14.793561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:14.820507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722d8a-593a-45f5-872d-1f41974b769a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.832546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.832546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.832546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.832546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.832999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.833089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1087/2172] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:14.833838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e3dbf-ce71-48d3-88bd-3d78e8e8fc9a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:14.836912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.869293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1086/2173] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:14.869293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464f10-88a5-449a-a598-2933e24e08af None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.869293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464f10-88a5-449a-a598-2933e24e08af None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.869293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464f10-88a5-449a-a598-2933e24e08af None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.870381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1087/2174] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:37:14.871149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da464f10-88a5-449a-a598-2933e24e08af None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:14.947269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc5b02-9015-44b9-88fb-869e56fb4168 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.975152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da464f10-88a5-449a-a598-2933e24e08af None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:14.984270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:14.985452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:14.985742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:14.986026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:14.986730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:14.988729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e385ac7-8bac-4e93-b806-5ea1c5e525c5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.009255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1088/2175] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:15.010779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.011062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.011363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.022714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.023532 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.023884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.024254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.025019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.027025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.031280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:15.032541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.033995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:15.048339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1088/2176] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:14 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:15.052858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.054853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.066380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829b85d-594c-4a98-8d97-adbbb3a5b390 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.071139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.072077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.072360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.072636 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] PATH_INFO: `/v3/users/a45d24d55fab4e9e852f2c65c771be17` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.073849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45d24d55fab4e9e852f2c65c771be17)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.075800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.098558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1089/2177] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:15.100141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.112701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c2c1a-4d73-460b-82eb-5995c799fb31 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.117690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.118356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.118639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.118919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.119684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.121278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.126785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45d24d55fab4e9e852f2c65c771be17'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.131798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1090/2178] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:15.134604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.138383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45d24d55fab4e9e852f2c65c771be17'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.139768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45d24d55fab4e9e852f2c65c771be17'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.142833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0b29a-20c6-4f9d-b748-17c61e9a53b0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.148092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.148739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.149020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.149298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.150018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.151646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.153295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45d24d55fab4e9e852f2c65c771be17'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.160155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45d24d55fab4e9e852f2c65c771be17'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.161859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1091/2179] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:15.165245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.165245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.169673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1089/2180] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:15 2022] DELETE /identity/v3/users/a45d24d55fab4e9e852f2c65c771be17 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:15.170480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.171791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.180194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68f579-2c48-479a-844c-c0dea4757a1a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.225823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:15.250687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b5d19-4ae0-4e7f-9485-e079ca529c27 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.252175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.252980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.253365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.253749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.256505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.256826 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.257731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.258212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.258584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.259592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.261251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.270525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.272914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.273697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1090/2181] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:15.274955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.277066 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.286534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1092/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:15.287696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.295615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01f235-b933-4322-b5c7-2048d2ce30e2 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.298266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e87918-3887-4e1e-a3ef-b17b5be8a2ab service placement] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.301295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.302824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.303195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.303672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.304436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.304620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.305568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.305914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.306251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] PATH_INFO: `/v3/projects/eede1d84f47b4ee1a25813195dfab3da/users/c3c77a90c230487fa6288cacc33e9166/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.306350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.321137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1091/2183] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:15.322307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.333716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3797f-aa8f-4e4a-9266-525328044ebe admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.336857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eede1d84f47b4ee1a25813195dfab3da, user_id=c3c77a90c230487fa6288cacc33e9166, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.339606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29fe62-774c-48d0-9442-95232b074372 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.344146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.345247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.345567 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.345851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] PATH_INFO: `/v3/projects/6d626a45f8a74c748b720ed295079fed/users/35dfd2a073164dac83acbf686ad7fb8b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.355175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1093/2184] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:15 2022] PUT /identity/v3/projects/eede1d84f47b4ee1a25813195dfab3da/users/c3c77a90c230487fa6288cacc33e9166/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:15.364907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d626a45f8a74c748b720ed295079fed, user_id=35dfd2a073164dac83acbf686ad7fb8b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.367266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c92ff-9ea5-46d4-ba37-07f19598ad58 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.367620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c92ff-9ea5-46d4-ba37-07f19598ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.368000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c92ff-9ea5-46d4-ba37-07f19598ad58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.368225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.382650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1092/2185] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:15 2022] PUT /identity/v3/projects/6d626a45f8a74c748b720ed295079fed/users/35dfd2a073164dac83acbf686ad7fb8b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:15.389413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.392574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.400885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9f5c92ff-9ea5-46d4-ba37-07f19598ad58 None None] MFA Rules not processed for user `c3c77a90c230487fa6288cacc33e9166`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:15.402210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9f5c92ff-9ea5-46d4-ba37-07f19598ad58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.424550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:15.472321 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f7874-41a5-4419-98e8-17363fe69db8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.477382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1094/2186] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:15.477673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.478540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.478823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.479090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] PATH_INFO: `/v3/projects/f923122fff9b4011a092bb70c4ec6ff3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.479372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699b1d1-cab2-4a6c-b807-68fdbdb8c32f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.479670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699b1d1-cab2-4a6c-b807-68fdbdb8c32f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.479944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699b1d1-cab2-4a6c-b807-68fdbdb8c32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.480637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f923122fff9b4011a092bb70c4ec6ff3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.483044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.508736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5699b1d1-cab2-4a6c-b807-68fdbdb8c32f None None] MFA Rules not processed for user `35dfd2a073164dac83acbf686ad7fb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:15.510409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5699b1d1-cab2-4a6c-b807-68fdbdb8c32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.537304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f923122fff9b4011a092bb70c4ec6ff3', 'request_id': 'req-b130ebeb-bb26-4f71-b36a-984aa402d437'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.543184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f923122fff9b4011a092bb70c4ec6ff3', 'request_id': 'req-b130ebeb-bb26-4f71-b36a-984aa402d437'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.552310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f923122fff9b4011a092bb70c4ec6ff3', 'request_id': 'req-b130ebeb-bb26-4f71-b36a-984aa402d437'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.570487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] The token cache is being invalidate because project f923122fff9b4011a092bb70c4ec6ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:15.570776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.571440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:15.573013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1093/2187] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:15 2022] DELETE /identity/v3/projects/f923122fff9b4011a092bb70c4ec6ff3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:15.573831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.575140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.582376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1095/2188] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:15.583830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.584120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.584395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.602373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:15.603665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.623503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:15.642409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130ebeb-bb26-4f71-b36a-984aa402d437 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.647955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.648632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.648917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.649197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.649892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.651884 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09d927-bd95-4b5d-97e6-330673eec861 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.665816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1096/2189] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:15.673889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.673889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.682133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1094/2190] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:15.682133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.682133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.682133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.700047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:15.701216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.732667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064440bb-abf0-4434-94df-3a07822652a2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.734201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:15.765672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1095/2191] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:15.769692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.779789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8925452-2fa6-4bc5-8687-4ec8467bf4e9 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.783550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.784197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.784475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.784577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.784746 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.785130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.785421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.785702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.786418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.786799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:15.788399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.812695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1096/2192] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/users => generated 298 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:15.812695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.821807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f2841-f08d-4038-81e8-b90adad21c2f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.826511 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.827158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.827481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.827761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.828486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:15.828795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.830448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.842953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1097/2193] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:15.843792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.853294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a430967-45be-40bb-ab7a-8ad52c1013fb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.856600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.858051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.858171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.858740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.859024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.859320 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.860067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.861689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.868023 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1097/2194] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:15.868659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.875815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1098/2195] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:15.876910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.879009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7230d4-ec71-4552-a5c7-328780cd8423 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.885987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.886657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.886941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.887219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.887944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.888451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baac118-c49d-4ed7-bf9a-696b51cb8e83 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:15.889912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.893721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:15.894610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:15.894895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:15.895203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] PATH_INFO: `/v3/projects/002f1885bedc42bd9862bf378beb43d4/users/5a0a86ab607d4f08ae2c0581c1156877/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:15.912833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002f1885bedc42bd9862bf378beb43d4, user_id=5a0a86ab607d4f08ae2c0581c1156877, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:15.915989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:15.919141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1098/2196] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:15.923263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.935348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1099/2197] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:15 2022] PUT /identity/v3/projects/002f1885bedc42bd9862bf378beb43d4/users/5a0a86ab607d4f08ae2c0581c1156877/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:15.939019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:15.947668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:16.020159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396c0e-03af-47c6-a86b-88a58dd45405 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.024916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b073f-7431-41e8-982b-c2a0f2d70cf1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.029262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.029951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.030523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.030689 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.031398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.031748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.032767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.033062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.033343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.033536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.034135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.035782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.050812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1100/2198] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:16.056424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.060370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1099/2199] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:15 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:16.064754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.068775 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b881d-15b0-40c3-9489-1f8ebcefa58c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.075397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.076363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.076671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.077592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] PATH_INFO: `/v3/projects/002f1885bedc42bd9862bf378beb43d4/users/5a0a86ab607d4f08ae2c0581c1156877/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.080156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=002f1885bedc42bd9862bf378beb43d4, user_id=5a0a86ab607d4f08ae2c0581c1156877, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.081505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f429a8-1b04-4302-9f3d-fcf09e1b517c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.082806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad00cc-5561-413a-823e-0b4eaa214e4c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.089491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.091071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.091480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.091888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.093119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.095057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.095423 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1101/2200] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:16 2022] PUT /identity/v3/projects/002f1885bedc42bd9862bf378beb43d4/users/5a0a86ab607d4f08ae2c0581c1156877/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:16.107124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.107357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1100/2201] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:16.107573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.109177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.110814 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.125147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f371c-2538-4aa0-88f0-2444e9589792 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.145910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] MFA Rules not processed for user `5a0a86ab607d4f08ae2c0581c1156877`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:16.147431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:16.199329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:16.218350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.219355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.219796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.220112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.220900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.222829 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.236337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1101/2202] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:16.240893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.242672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1102/2203] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:16.252987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2257-2aa0-43aa-aafb-ea25f0bcce60 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.257906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.258900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.259832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.260111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.260511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] PATH_INFO: `/v3/projects/78534de78c2b49a6b194a723fadf4a41/users/891a6b2e1fea4a5aa3ed01a625b280b9/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.279973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78534de78c2b49a6b194a723fadf4a41, user_id=891a6b2e1fea4a5aa3ed01a625b280b9, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.283320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55d2de-2d59-4da4-b860-43f5d11fc356 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.299390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1102/2204] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:16 2022] PUT /identity/v3/projects/78534de78c2b49a6b194a723fadf4a41/users/891a6b2e1fea4a5aa3ed01a625b280b9/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:16.309478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.309763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.310039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.322431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f93f8f-6b0b-43a6-a1a6-96aebb6bc5fd None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.338882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] MFA Rules not processed for user `891a6b2e1fea4a5aa3ed01a625b280b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:16.340125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:16.373411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.374063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.374370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.374648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.376521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:16.398132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:16.408991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1103/2205] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:16.420159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:16.421540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:16.433509 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f2e14-7574-45c6-8207-3030bbda0f5c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:16.437192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:16.437874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:16.438161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:16.438439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:16.443481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.447111 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd41b-a413-4ad3-8488-366eccd55e64 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.448556 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:16.449994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:16.456916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1103/2206] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:16.460518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1104/2207] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:37:17.343388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:17.343924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:17.343924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:17.362286 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] MFA Rules not processed for user `35dfd2a073164dac83acbf686ad7fb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:17.363524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:17.407671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:17.419118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1104/2208] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:17.993918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:18.002677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc813ee-afc0-4a05-955e-68b167bdcccf service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:18.006707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:18.007449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:18.007801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:18.008206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:18.009910 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:18.078621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:18.080333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:18.091234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1105/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:18.751555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:18.762862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813f892-4fda-4e4b-9d1d-b469d58cd544 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:18.767345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:18.767988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:18.768265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:18.768545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:18.843187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:18.844680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cdaa7-1db3-4d23-b766-5baca48dee3f service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:18.855344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1105/2210] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:22.931219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:23.006830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef3f5c-2543-4add-9e78-3b8ffded2ade service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:23.011728 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:23.013347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:23.013777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:23.014067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] PATH_INFO: `/v3/users/b6251c77d2ee437791dcc35597fba5f1` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:23.015571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6251c77d2ee437791dcc35597fba5f1)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:23.017994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:23.066396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6251c77d2ee437791dcc35597fba5f1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.073757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6251c77d2ee437791dcc35597fba5f1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.085664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6251c77d2ee437791dcc35597fba5f1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.086301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6251c77d2ee437791dcc35597fba5f1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.093161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6251c77d2ee437791dcc35597fba5f1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.095305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:23.100640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1106/2211] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:22 2022] DELETE /identity/v3/users/b6251c77d2ee437791dcc35597fba5f1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:23.105620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:23.106097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:23.106190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:23.123484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:23.124672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:23.187568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1106/2212] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:23.197876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:23.199559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:23.265607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e3415-086a-4826-900b-8f2eda47c13d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:23.269709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:23.270323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:23.270583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:23.271663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:23.281606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:23.283059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:23.293272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1107/2213] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:23.655935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:23.657358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:23.673271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcf6cb-cf59-4b9d-896d-47a7b99cd739 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:23.680198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:23.681051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:23.681350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:23.681593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] PATH_INFO: `/v3/projects/53edaab7a665412c954dddcfab7ba106` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:23.682828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53edaab7a665412c954dddcfab7ba106)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:23.684766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:23.731715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53edaab7a665412c954dddcfab7ba106', 'request_id': 'req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.737132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53edaab7a665412c954dddcfab7ba106', 'request_id': 'req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.744634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53edaab7a665412c954dddcfab7ba106', 'request_id': 'req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.756518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] The token cache is being invalidate because project 53edaab7a665412c954dddcfab7ba106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:23.756807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.757471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.759024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1107/2214] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:23 2022] DELETE /identity/v3/projects/53edaab7a665412c954dddcfab7ba106 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:23.763147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:23.773616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8276545-24a1-494a-a20f-9796aa5d654f service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:23.828650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:23.829517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:23.829805 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:23.830077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] PATH_INFO: `/v3/users/c3c77a90c230487fa6288cacc33e9166` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:23.831281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c77a90c230487fa6288cacc33e9166)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:23.833778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:23.877724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c77a90c230487fa6288cacc33e9166'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.886385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c77a90c230487fa6288cacc33e9166'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.899024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c77a90c230487fa6288cacc33e9166'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.899692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c77a90c230487fa6288cacc33e9166'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.906848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c77a90c230487fa6288cacc33e9166'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:23.912770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1108/2215] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:23 2022] DELETE /identity/v3/users/c3c77a90c230487fa6288cacc33e9166 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:24.261376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:24.263216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:24.281516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a5bf-a12e-4b25-8e85-d60b14cb5992 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:24.293906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:24.347669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:24.348595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:24.349038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:24.349347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] PATH_INFO: `/v3/projects/eede1d84f47b4ee1a25813195dfab3da` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:24.350588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eede1d84f47b4ee1a25813195dfab3da)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:24.352574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:24.400054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eede1d84f47b4ee1a25813195dfab3da', 'request_id': 'req-155bea85-089f-4115-b1ad-196b004eb37e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.406051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eede1d84f47b4ee1a25813195dfab3da', 'request_id': 'req-155bea85-089f-4115-b1ad-196b004eb37e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.414135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eede1d84f47b4ee1a25813195dfab3da', 'request_id': 'req-155bea85-089f-4115-b1ad-196b004eb37e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.425665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] The token cache is being invalidate because project eede1d84f47b4ee1a25813195dfab3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:24.425951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.426587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.428710 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1108/2216] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:24 2022] DELETE /identity/v3/projects/eede1d84f47b4ee1a25813195dfab3da => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:24.432775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:24.434176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:24.447953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783db79-3989-4d4d-9f66-0fa9d799fa67 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:24.466491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:24.508619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:24.509485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:24.509759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:24.510025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] PATH_INFO: `/v3/users/5a0a86ab607d4f08ae2c0581c1156877` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:24.511353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0a86ab607d4f08ae2c0581c1156877)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:24.513348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:24.564804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0a86ab607d4f08ae2c0581c1156877'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.573368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0a86ab607d4f08ae2c0581c1156877'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.590557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0a86ab607d4f08ae2c0581c1156877'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.592649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0a86ab607d4f08ae2c0581c1156877'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.601380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c629ab65-5119-412c-bdbe-a14507414a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0a86ab607d4f08ae2c0581c1156877'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:24.608772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1109/2217] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:24 2022] DELETE /identity/v3/users/5a0a86ab607d4f08ae2c0581c1156877 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:24.884726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:24.886202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:24.902340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bea85-089f-4115-b1ad-196b004eb37e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:24.959713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:24.960586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:24.960934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:24.961218 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] PATH_INFO: `/v3/projects/002f1885bedc42bd9862bf378beb43d4` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:24.962438 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=002f1885bedc42bd9862bf378beb43d4)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:24.964520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.007373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:25.010770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002f1885bedc42bd9862bf378beb43d4', 'request_id': 'req-0fa32517-d873-4919-876d-d5c89beb1ec4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.017181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002f1885bedc42bd9862bf378beb43d4', 'request_id': 'req-0fa32517-d873-4919-876d-d5c89beb1ec4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.025161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002f1885bedc42bd9862bf378beb43d4', 'request_id': 'req-0fa32517-d873-4919-876d-d5c89beb1ec4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.036238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] The token cache is being invalidate because project 002f1885bedc42bd9862bf378beb43d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:25.036609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.037237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.038769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1109/2218] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:24 2022] DELETE /identity/v3/projects/002f1885bedc42bd9862bf378beb43d4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:25.044218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034df1-1ce8-455c-bba5-c24ab34e28aa None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.044502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034df1-1ce8-455c-bba5-c24ab34e28aa None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.044777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7034df1-1ce8-455c-bba5-c24ab34e28aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.062701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e7034df1-1ce8-455c-bba5-c24ab34e28aa None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:25.063980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e7034df1-1ce8-455c-bba5-c24ab34e28aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.121836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1110/2219] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:25.178625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.180178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.188348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.189342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.189342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.193809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa32517-d873-4919-876d-d5c89beb1ec4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.198970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.200562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.200562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.200562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] PATH_INFO: `/v3/users/891a6b2e1fea4a5aa3ed01a625b280b9` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.201648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=891a6b2e1fea4a5aa3ed01a625b280b9)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.204099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.211436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:25.212503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.223434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1111/2220] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:25.226873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.245937 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88992a-fe64-48a1-b411-f93e1f7e41b3 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.253197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.253934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.254322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.254617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.256058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:25.256379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.258989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.280394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1112/2221] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:25.281568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.289428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891a6b2e1fea4a5aa3ed01a625b280b9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.294776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc96c7-d8b7-42eb-bf4d-5dc19ef0d1c4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.300762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.301158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891a6b2e1fea4a5aa3ed01a625b280b9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.301670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.302005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.302346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.303103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891a6b2e1fea4a5aa3ed01a625b280b9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.303330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.305489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.326339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891a6b2e1fea4a5aa3ed01a625b280b9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.339094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:25.339744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-e4b351d9-301f-453a-b6be-27522ac9f60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891a6b2e1fea4a5aa3ed01a625b280b9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:25.352854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1110/2222] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:25 2022] DELETE /identity/v3/users/891a6b2e1fea4a5aa3ed01a625b280b9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:25.353067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1113/2223] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:25.355632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.355756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.356047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.356617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.358903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.385338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:25.387323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.439547 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f15b5-dd00-40a2-90ef-c4ab36cba4d7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.466001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1111/2224] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:25.476927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.478433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.511902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.512596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.513389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.513389 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.513828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.515779 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.535255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:25.539335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1114/2225] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:25.541334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.542242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a063c-0520-4c30-9ecc-2df2b593925e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.548260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.548994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.549484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.549835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.551755 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0f5dd-41c2-4829-90b2-4e834ac0418e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.556574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.557153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.557435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.557713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.558428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.560026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.560139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.561439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.570343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1115/2226] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:25.573172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1112/2227] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:37:25.574497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.584705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b978d-9c63-43ec-ba74-a22b6ddd5344 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.589311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.589961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.590246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.590515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.591259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.592869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce7f4b-391f-495a-a4d2-7fb934d42784 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.604026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1116/2228] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:25.608475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.617821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fba16-d75d-43ea-8d54-451398f8f212 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.623190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.624337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.624626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.624903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] PATH_INFO: `/v3/projects/f4e4dfcce8754d97973892db79a48859/users/b1158edcff7747e19e369a0d01227798/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.645455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e4dfcce8754d97973892db79a48859, user_id=b1158edcff7747e19e369a0d01227798, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.649136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.673722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1113/2229] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:25 2022] PUT /identity/v3/projects/f4e4dfcce8754d97973892db79a48859/users/b1158edcff7747e19e369a0d01227798/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:25.687221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.688218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.688605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.728274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] MFA Rules not processed for user `b1158edcff7747e19e369a0d01227798`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:25.729864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.850923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1117/2230] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:25.864800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.867003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.930184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:25.931715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:25.938199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:25.952195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee300fa-32a6-41bf-999f-8e67693cffcf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:25.959213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:25.960733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:25.961013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:25.961432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:25.973644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:25.975242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba582a-b3de-44c1-9c74-ac96aea87c52 service glance] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:25.984689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1114/2231] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:26.009293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11e770-e67c-4209-a086-b01ddc79891a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.014357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.015235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.015540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.015811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] PATH_INFO: `/v3/projects/78534de78c2b49a6b194a723fadf4a41` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.016993 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78534de78c2b49a6b194a723fadf4a41)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.018896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.066362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78534de78c2b49a6b194a723fadf4a41', 'request_id': 'req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:26.074524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78534de78c2b49a6b194a723fadf4a41', 'request_id': 'req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:26.080268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78534de78c2b49a6b194a723fadf4a41', 'request_id': 'req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:26.091898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] The token cache is being invalidate because project 78534de78c2b49a6b194a723fadf4a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:26.092112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:26.092783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-60d37aa0-ecac-4d47-bb5d-f4ef47fa3482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:26.094627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1118/2232] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:25 2022] DELETE /identity/v3/projects/78534de78c2b49a6b194a723fadf4a41 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:26.100184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.100403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.100683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.119748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:26.120999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:26.191600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1115/2233] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:26.199150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.199479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.199786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.219343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:26.220554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:26.229267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1119/2234] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:26.234136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.245691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674679-f73d-4f0e-abb2-c4f64c73abe8 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.250719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.251454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.251755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.252157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.252946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:26.253292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.254855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.276161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1116/2235] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:26.277626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.289148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c86dc-4044-47ce-8379-1f650a8d1f35 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.294334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.294988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.295271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.295574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.296268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.298172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.325210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1120/2236] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:26 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:26.329416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.340105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b74d8-46c9-4161-8841-8cc54f606804 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.397816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:26.407164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.408144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.408601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.408970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.409766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.412087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.457081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1117/2237] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:26 2022] POST /identity/v3/users => generated 303 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:26.466664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.485946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4593d-7ee2-47c0-a173-eda68e506a75 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.499239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.500473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.501072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.501678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.512562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.521293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.542225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1121/2238] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:26.544077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.565387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f901-a1ed-4b55-9a38-333f0d49862c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.574784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.576032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.577020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.577127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.578586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.581852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37fce37-d1d8-4222-bf41-a82fb810aab3 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.610290 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1118/2239] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:26.613149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.624659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53354c04-930f-45e2-835b-e2add73520ed admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.629564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.631110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.631483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.631821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] PATH_INFO: `/v3/projects/2b1cce39aec143f5aaec0f1d2cfac998/users/71fbc8bf5c4742ab95c3af4c772a3d58/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.647303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b1cce39aec143f5aaec0f1d2cfac998, user_id=71fbc8bf5c4742ab95c3af4c772a3d58, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.650066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.663431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1122/2240] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:26 2022] PUT /identity/v3/projects/2b1cce39aec143f5aaec0f1d2cfac998/users/71fbc8bf5c4742ab95c3af4c772a3d58/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:26.668807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbb71-ebb4-451d-919f-64e90bbe0422 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.669094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbb71-ebb4-451d-919f-64e90bbe0422 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.669369 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dbb71-ebb4-451d-919f-64e90bbe0422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.687679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ac9dbb71-ebb4-451d-919f-64e90bbe0422 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:26.688879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ac9dbb71-ebb4-451d-919f-64e90bbe0422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:26.750656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1119/2241] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:26.760238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:26.761938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:26.796524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:26.821603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73297d2-72bd-4aa6-82ef-a913e2d7c3a8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:26.825274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:26.825913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:26.826194 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:26.826467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:26.835525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:26.836863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:26.846595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1123/2242] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:27.543134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:27.543652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:27.543737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:27.585889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] MFA Rules not processed for user `71fbc8bf5c4742ab95c3af4c772a3d58`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:27.587329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:27.681571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1120/2243] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:27.686055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:27.697828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b51469-6eca-4ff4-987d-901174218561 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:27.704530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:27.705587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:27.705933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:27.706278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:27.707109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:27.709267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:27.759179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1124/2244] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:27 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:27.763264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:27.779866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336a10c-0c2e-4e7a-837b-05e18656ff6a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:27.860722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:27.861485 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:27.861866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:27.862235 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:27.862894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:27.865568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:27.874468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:27.900277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1121/2245] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:27 2022] POST /identity/v3/users => generated 308 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:27.901576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:27.914119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7decbb83-d8f9-49c2-986e-575a907d5c09 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:27.920552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:27.921760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:27.922049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:27.922325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:27.923244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:27.924957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:27.944255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:27.951368 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1125/2246] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:27.956202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbdb83-b69f-476a-a55a-2063a8f48102 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:27.963615 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:27.964589 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:27.964905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:27.965155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] PATH_INFO: `/v3/projects/f4b4664f4e47459db705cd31e3cf451b/users/043d8494eb8447448408a5b58daa618e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:27.986219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b4664f4e47459db705cd31e3cf451b, user_id=043d8494eb8447448408a5b58daa618e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:27.988985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:28.005302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1122/2247] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:27 2022] PUT /identity/v3/projects/f4b4664f4e47459db705cd31e3cf451b/users/043d8494eb8447448408a5b58daa618e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:28.008999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:28.019585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78381d1b-8a25-4e54-a009-ab0eeeb18fa7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:28.100007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:28.101206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:28.101564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:28.101854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:28.102789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:28.104690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4af74c-2d6a-4067-8e00-aca5cfcddd79 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:28.121328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1126/2248] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:28.122791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:28.148752 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a96c4-b72b-4c83-8e7d-79419143123e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:28.154671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:28.155767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:28.156441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:28.156950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] PATH_INFO: `/v3/projects/f4b4664f4e47459db705cd31e3cf451b/users/043d8494eb8447448408a5b58daa618e/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:28.159747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b4664f4e47459db705cd31e3cf451b, user_id=043d8494eb8447448408a5b58daa618e, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:28.163458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:28.179534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1123/2249] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:28 2022] PUT /identity/v3/projects/f4b4664f4e47459db705cd31e3cf451b/users/043d8494eb8447448408a5b58daa618e/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:28.922622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:28.923114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:28.923387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:28.951645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] MFA Rules not processed for user `043d8494eb8447448408a5b58daa618e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:28.952850 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:29.028975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1127/2250] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:30.141472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:30.211683 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1799f0f-cd95-4c96-8040-018537d1364b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:30.216443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:30.217274 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:30.217561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:30.217839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] PATH_INFO: `/v3/users/71fbc8bf5c4742ab95c3af4c772a3d58` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:30.219050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fbc8bf5c4742ab95c3af4c772a3d58)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:30.221011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:30.277207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fbc8bf5c4742ab95c3af4c772a3d58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.284301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fbc8bf5c4742ab95c3af4c772a3d58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.285333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fbc8bf5c4742ab95c3af4c772a3d58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.299228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fbc8bf5c4742ab95c3af4c772a3d58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.306424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fbc8bf5c4742ab95c3af4c772a3d58'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.312744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1124/2251] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:30 2022] DELETE /identity/v3/users/71fbc8bf5c4742ab95c3af4c772a3d58 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:30.494214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:30.510478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d2ce-e8b8-475b-9630-a87f50ded3d9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:30.530735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:30.572570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:30.573358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:30.573644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:30.573921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] PATH_INFO: `/v3/projects/2b1cce39aec143f5aaec0f1d2cfac998` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:30.575115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b1cce39aec143f5aaec0f1d2cfac998)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:30.577070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:30.623365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b1cce39aec143f5aaec0f1d2cfac998', 'request_id': 'req-c1c04855-1eba-49cb-a7b3-002f4edf5abf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.632279 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b1cce39aec143f5aaec0f1d2cfac998', 'request_id': 'req-c1c04855-1eba-49cb-a7b3-002f4edf5abf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.637766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b1cce39aec143f5aaec0f1d2cfac998', 'request_id': 'req-c1c04855-1eba-49cb-a7b3-002f4edf5abf'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.650049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] The token cache is being invalidate because project 2b1cce39aec143f5aaec0f1d2cfac998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:30.650339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.651026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.652820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1128/2252] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:30 2022] DELETE /identity/v3/projects/2b1cce39aec143f5aaec0f1d2cfac998 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:30.656865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:30.659989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:30.673413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036929d-5511-49b1-a05b-e2450b0f3b80 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:30.707806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:30.744123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:30.745153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:30.745545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:30.745919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] PATH_INFO: `/v3/users/043d8494eb8447448408a5b58daa618e` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:30.747474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=043d8494eb8447448408a5b58daa618e)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:30.749555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:30.819424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043d8494eb8447448408a5b58daa618e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.824701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043d8494eb8447448408a5b58daa618e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.825319 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043d8494eb8447448408a5b58daa618e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.837363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043d8494eb8447448408a5b58daa618e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.844098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-c0dedb71-f5d8-41c0-868b-dad8f6248e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043d8494eb8447448408a5b58daa618e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:30.850035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1125/2253] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:30 2022] DELETE /identity/v3/users/043d8494eb8447448408a5b58daa618e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:31.040579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.042132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:31.057994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04855-1eba-49cb-a7b3-002f4edf5abf admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.120210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.121067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.121346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.121618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] PATH_INFO: `/v3/projects/f4b4664f4e47459db705cd31e3cf451b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.122785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b4664f4e47459db705cd31e3cf451b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.124719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.177662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b4664f4e47459db705cd31e3cf451b', 'request_id': 'req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:31.181385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:31.185699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b4664f4e47459db705cd31e3cf451b', 'request_id': 'req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:31.193655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b4664f4e47459db705cd31e3cf451b', 'request_id': 'req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:31.204570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] The token cache is being invalidate because project f4b4664f4e47459db705cd31e3cf451b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:31.204860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:31.205495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6c0b646d-7437-4d65-9d2c-1b3d74eddf4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:31.207038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1129/2254] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:31 2022] DELETE /identity/v3/projects/f4b4664f4e47459db705cd31e3cf451b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:31.212841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.213309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.213429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.232373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:31.233593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:31.296799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1126/2255] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:31.305563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.305881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.306154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.325436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:31.326700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:31.334938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1130/2256] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:31.339371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.350112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2a922-6672-4ef5-ad64-0750b91fcf63 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.354802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.355498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.355799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.356118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.356847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:31.357154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.358788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.371717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1127/2257] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:31.375965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.385415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc558c-798a-4cf0-867b-63d18d697f78 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.390074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.390714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.390989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.391261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.391987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.393864 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.422095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1131/2258] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:31.426347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.437336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f42bc-1150-4880-9f00-09abe326bb6e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.482658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:31.501416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.502487 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.502915 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.503196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.503944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.506599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.532955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1128/2259] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:31.538273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.549930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196c845-3e64-40d8-a418-93dcfea5e81c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.555870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.556610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.556907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.557175 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.557992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.559733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.573487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1132/2260] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:31.577527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.591061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577dd86-1c67-4c84-9355-3a92bb972a72 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.595844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.596505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.596789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.597062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.597789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.600434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91095a8e-8054-4745-9c07-7b782a180b35 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.614473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1129/2261] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:31.618677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.628685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925c4-d06e-454a-bc18-87f0c12761c8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.633354 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.634221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.634499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.634771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] PATH_INFO: `/v3/projects/ffa4603027604a6bb25f8cdd1b012421/users/b8c24c22a28d4ca7a19af021255e8d63/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.649968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa4603027604a6bb25f8cdd1b012421, user_id=b8c24c22a28d4ca7a19af021255e8d63, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.652410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.678663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1133/2262] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:31 2022] PUT /identity/v3/projects/ffa4603027604a6bb25f8cdd1b012421/users/b8c24c22a28d4ca7a19af021255e8d63/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:31.684465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.684465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.684465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.707212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] MFA Rules not processed for user `b8c24c22a28d4ca7a19af021255e8d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:31.708965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:31.774989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1130/2263] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:37:31.779708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.855886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a2ccd-5474-465d-9127-298cc2b0b067 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.860627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.861281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.861566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.861946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.862730 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.865110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.893918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1134/2264] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:31.898715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:31.909779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7bd33-08db-41c4-8466-0603fb6f9c4e None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:31.971881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:31.972571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:31.975814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:31.975962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:31.976697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:31.978652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:31.999004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1131/2265] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:31 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:32.002851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.014338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1d76d-95db-4e71-b961-9be841a8d858 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.019409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.020127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.020416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.020771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.022138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.023848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.035157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1135/2266] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:32.039406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.049538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54a428-b6e1-4087-9a37-fb9c298d2c1f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.054746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.055642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.055926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.056202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] PATH_INFO: `/v3/projects/ad28732314354530be40559314c64db7/users/e2cefbd6d04f47b39d04a5a1c62c9d3f/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.071652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad28732314354530be40559314c64db7, user_id=e2cefbd6d04f47b39d04a5a1c62c9d3f, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.074312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.087281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1132/2267] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:32 2022] PUT /identity/v3/projects/ad28732314354530be40559314c64db7/users/e2cefbd6d04f47b39d04a5a1c62c9d3f/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:32.091428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.103619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2c34e-4cd2-44e7-bfd0-f1c4f6175124 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.151634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:32.161303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.162359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.162812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.163242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.164204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.166072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4debf4e-4d4c-40b0-b9a3-fc7b119281d7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.176273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1136/2268] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:32.180623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.191791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc48d3-0ce1-4937-960b-421b850ccd2f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.197965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.199224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.199556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.199840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] PATH_INFO: `/v3/projects/ad28732314354530be40559314c64db7/users/e2cefbd6d04f47b39d04a5a1c62c9d3f/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.201878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad28732314354530be40559314c64db7, user_id=e2cefbd6d04f47b39d04a5a1c62c9d3f, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.204585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.216536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1133/2269] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:32 2022] PUT /identity/v3/projects/ad28732314354530be40559314c64db7/users/e2cefbd6d04f47b39d04a5a1c62c9d3f/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:32.226573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.226865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.227149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.256045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] MFA Rules not processed for user `e2cefbd6d04f47b39d04a5a1c62c9d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:32.257316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:32.333513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1137/2270] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:32.344580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.346612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:32.352799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:32.420695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92e33a-a06d-4b45-ad12-b4f906d0a01d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.429371 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.430037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.430323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.430598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.448424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.450040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091332fc-d1eb-4615-aad6-ec0e29af8a83 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.467994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1134/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:32.532406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.597306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2ded6-52a4-4202-825d-ba0bf6ce7f2e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.601847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.602714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.602991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.603261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] PATH_INFO: `/v3/users/b8c24c22a28d4ca7a19af021255e8d63` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.604485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c24c22a28d4ca7a19af021255e8d63)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.606386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.660897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c24c22a28d4ca7a19af021255e8d63'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:32.670924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c24c22a28d4ca7a19af021255e8d63'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:32.685499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c24c22a28d4ca7a19af021255e8d63'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:32.686293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c24c22a28d4ca7a19af021255e8d63'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:32.692185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:32.696740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c24c22a28d4ca7a19af021255e8d63'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:32.706404 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1138/2272] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:32 2022] DELETE /identity/v3/users/b8c24c22a28d4ca7a19af021255e8d63 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:32.711410 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.711781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.715196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.737760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:32.738985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:32.816547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1135/2273] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:32.826830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:32.828290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:32.890580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b33122-5bf1-4095-a756-b8c5cf2d06cd admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:32.894332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:32.895009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:32.895313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:32.895589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:32.905369 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:32.906830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:32.916775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1139/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:33.206005 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:33.207536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:33.220087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a8e5b-ce90-4d1d-8dcc-e723550e391d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:33.225070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:33.225945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:33.226897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:33.227544 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] PATH_INFO: `/v3/projects/ffa4603027604a6bb25f8cdd1b012421` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:33.228802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa4603027604a6bb25f8cdd1b012421)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:33.231146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:33.288409 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa4603027604a6bb25f8cdd1b012421', 'request_id': 'req-7921e737-6ebf-4908-a2d4-577e76b90806'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.294287 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa4603027604a6bb25f8cdd1b012421', 'request_id': 'req-7921e737-6ebf-4908-a2d4-577e76b90806'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.301561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa4603027604a6bb25f8cdd1b012421', 'request_id': 'req-7921e737-6ebf-4908-a2d4-577e76b90806'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.313586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] The token cache is being invalidate because project ffa4603027604a6bb25f8cdd1b012421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:33.313875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.314547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.316214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1136/2275] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:33 2022] DELETE /identity/v3/projects/ffa4603027604a6bb25f8cdd1b012421 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:33.320555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:33.331445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff97ca4-5e55-4249-a977-d261eb9fd53f service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:33.389442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:33.390217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:33.390500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:33.390774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] PATH_INFO: `/v3/users/e2cefbd6d04f47b39d04a5a1c62c9d3f` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:33.392016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2cefbd6d04f47b39d04a5a1c62c9d3f)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:33.393969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:33.454066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2cefbd6d04f47b39d04a5a1c62c9d3f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.461138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2cefbd6d04f47b39d04a5a1c62c9d3f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.473616 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2cefbd6d04f47b39d04a5a1c62c9d3f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.474969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2cefbd6d04f47b39d04a5a1c62c9d3f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.482218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-32516dbc-05fe-40cd-874e-d08dc425fad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2cefbd6d04f47b39d04a5a1c62c9d3f'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.488748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1140/2276] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:33 2022] DELETE /identity/v3/users/e2cefbd6d04f47b39d04a5a1c62c9d3f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:33.759052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:33.760996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:33.774504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921e737-6ebf-4908-a2d4-577e76b90806 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:33.804451 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:33.838267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:33.839139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:33.839456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:33.839737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] PATH_INFO: `/v3/projects/ad28732314354530be40559314c64db7` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:33.840923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad28732314354530be40559314c64db7)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:33.842821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:33.891746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad28732314354530be40559314c64db7', 'request_id': 'req-25401483-2fb2-4e8c-a8b9-3c068d86ca26'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.901811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad28732314354530be40559314c64db7', 'request_id': 'req-25401483-2fb2-4e8c-a8b9-3c068d86ca26'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.913042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad28732314354530be40559314c64db7', 'request_id': 'req-25401483-2fb2-4e8c-a8b9-3c068d86ca26'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.930664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] The token cache is being invalidate because project ad28732314354530be40559314c64db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:33.930955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.931648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-25401483-2fb2-4e8c-a8b9-3c068d86ca26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:33.935323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1137/2277] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:33 2022] DELETE /identity/v3/projects/ad28732314354530be40559314c64db7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:33.938402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:33.938883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:33.939073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:33.958300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:33.959586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:33.976124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:34.027224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1141/2278] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:34.036154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.036484 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.036763 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.055630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:34.056893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:34.066842 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1138/2279] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:34.072083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.081953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ac7c-5499-4a5c-ae2d-da7999931ebe None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.086626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.087301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.087595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.087872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.088605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:34.088912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.090536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.102612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1142/2280] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:37:34.107895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.118454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783c0a3-ba4d-44b9-9284-b5c0c11a1aec None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.123098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.123800 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.124113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.124384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.125065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.126945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.154430 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1139/2281] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:34.158463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.169578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b088d-b3bb-4ce5-803a-1b5b65b49e28 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.228098 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.228973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.229353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.229724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.230558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.233451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.256328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1143/2282] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:34.261015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.274922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27fba-7d13-42dd-a737-24317d29d967 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.279914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.280616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.280896 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.281170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.281917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.283630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.296398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1140/2283] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:34.300587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.312029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc859-dc1d-4ff7-80f6-bca95f412f89 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.316894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.317586 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.317871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.318146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.319013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.320703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff44b05-23b1-4e48-b73b-885197dfff20 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.332097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1144/2284] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:34.336350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.346616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a2ac1-62d2-4ac0-bf42-d055b267e1e9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.352074 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.352987 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.353273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.353555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] PATH_INFO: `/v3/projects/27e9c129119a4fc2965b7573e41396f8/users/953fc521ce464cebacf7628ece4044c0/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.369604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e9c129119a4fc2965b7573e41396f8, user_id=953fc521ce464cebacf7628ece4044c0, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.372309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.388237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1141/2285] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:34 2022] PUT /identity/v3/projects/27e9c129119a4fc2965b7573e41396f8/users/953fc521ce464cebacf7628ece4044c0/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:34.398496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.398834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.399121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.427909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] MFA Rules not processed for user `953fc521ce464cebacf7628ece4044c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:34.429143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:34.514748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1145/2286] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:34.519355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.576906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:34.585051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0e84-24a1-49be-b72c-7843dc2d0e3f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.592126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.592797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.593080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.593354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.594069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.596085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.623203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1142/2287] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:34.627431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.638301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e52c50-d04d-448f-b4cd-9767f9b01c16 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.646363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:34.699058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.699668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.699946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.700213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.700895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.702798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.722281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1146/2288] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:34.727055 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.737909 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413ffce-8536-4690-8963-74db2017388b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.742981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.743726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.744010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.744275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.744996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.747736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.762578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1143/2289] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:34.766831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.777377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2568-6777-40b0-b7c6-3471f3c2470d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.782106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.783008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.783310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.783601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] PATH_INFO: `/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34/users/d4fbc97ea09a4041b46b99a4639ae83b/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.800739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0599b183f7fa4aa7b6bad38b25f91f34, user_id=d4fbc97ea09a4041b46b99a4639ae83b, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.804099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.817951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1147/2290] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:34 2022] PUT /identity/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34/users/d4fbc97ea09a4041b46b99a4639ae83b/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:34.822123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.832284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b81601-390f-4d23-8224-722bdae503af admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.887633 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.890820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.891546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.891829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.892594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.894352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da8a8f-f73c-423b-96f2-a610100e2ad9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.907361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1144/2291] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:34.911664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:34.922110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98679b-6fac-4c35-9e77-929e7b83dd0a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:34.926791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:34.927966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.928255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.928531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] PATH_INFO: `/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34/users/d4fbc97ea09a4041b46b99a4639ae83b/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.930512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0599b183f7fa4aa7b6bad38b25f91f34, user_id=d4fbc97ea09a4041b46b99a4639ae83b, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:34.933207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:34.945111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1148/2292] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:34 2022] PUT /identity/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34/users/d4fbc97ea09a4041b46b99a4639ae83b/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:34.955497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:34.955794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:34.956072 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:34.987027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] MFA Rules not processed for user `d4fbc97ea09a4041b46b99a4639ae83b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:34.988301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:35.065125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1145/2293] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:35.076841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:35.078740 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:35.142044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb9431-aaf2-4796-86af-daa2624f2b50 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:35.145783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:35.146989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:35.147278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:35.147581 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:35.156802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:35.158238 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:35.168355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1149/2294] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:35.213778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:35.215169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:35.227528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee21547-f4af-4242-a102-9ea9a35882f4 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:35.231213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:35.231889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:35.232176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:35.232460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:35.282302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:35.293534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:35.294978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476e67-11ef-4cfc-84ff-0e60450a5e05 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:35.304502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1146/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:40.719110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:40.780293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213484b7-95df-4933-b2f3-7835daedbf3c service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:40.785192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:40.786072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:40.786360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:40.786639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] PATH_INFO: `/v3/users/953fc521ce464cebacf7628ece4044c0` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:40.787941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953fc521ce464cebacf7628ece4044c0)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:40.790043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:40.799273 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:40.845890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953fc521ce464cebacf7628ece4044c0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:40.855259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953fc521ce464cebacf7628ece4044c0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:40.869421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953fc521ce464cebacf7628ece4044c0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:40.870128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953fc521ce464cebacf7628ece4044c0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:40.877282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953fc521ce464cebacf7628ece4044c0'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:40.885101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1150/2296] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:40 2022] DELETE /identity/v3/users/953fc521ce464cebacf7628ece4044c0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:40.889820 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:40.890276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:40.890398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:40.907794 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:40.908995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:40.974853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1147/2297] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:40.985619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:40.987124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:41.044887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e542d9e-d55e-47d9-9310-8e329c9f5df9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:41.048612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:41.049313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:41.049597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:41.049874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:41.059625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:41.061054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:41.070582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1151/2298] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:41.398466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:41.400143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:41.412751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f0002-0112-451f-950a-7bae2e8eeb25 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:41.418083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:41.424376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:41.424672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:41.424959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] PATH_INFO: `/v3/projects/27e9c129119a4fc2965b7573e41396f8` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:41.426291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e9c129119a4fc2965b7573e41396f8)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:41.429969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:41.485415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e9c129119a4fc2965b7573e41396f8', 'request_id': 'req-7ad37471-52a2-4c18-a78e-c547cb957579'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.491806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e9c129119a4fc2965b7573e41396f8', 'request_id': 'req-7ad37471-52a2-4c18-a78e-c547cb957579'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.500386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e9c129119a4fc2965b7573e41396f8', 'request_id': 'req-7ad37471-52a2-4c18-a78e-c547cb957579'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.513767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] The token cache is being invalidate because project 27e9c129119a4fc2965b7573e41396f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:41.514054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.514704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.516305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1148/2299] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:41 2022] DELETE /identity/v3/projects/27e9c129119a4fc2965b7573e41396f8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:41.520283 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:41.530210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950c4a-d6f3-47a0-b525-fe6c135c3b01 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:41.588854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:41.589738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:41.590020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:41.590294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] PATH_INFO: `/v3/users/d4fbc97ea09a4041b46b99a4639ae83b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:41.592108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fbc97ea09a4041b46b99a4639ae83b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:41.594119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:41.628748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:41.644289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fbc97ea09a4041b46b99a4639ae83b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.651590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fbc97ea09a4041b46b99a4639ae83b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.664710 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fbc97ea09a4041b46b99a4639ae83b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.665371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fbc97ea09a4041b46b99a4639ae83b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.672660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c14d34b8-ac32-472a-9196-7950b19a17b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fbc97ea09a4041b46b99a4639ae83b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:41.679202 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1152/2300] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:41 2022] DELETE /identity/v3/users/d4fbc97ea09a4041b46b99a4639ae83b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:41.960018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:41.961582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:41.977401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad37471-52a2-4c18-a78e-c547cb957579 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.039627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.040437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.041016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.041247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] PATH_INFO: `/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.042465 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0599b183f7fa4aa7b6bad38b25f91f34)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.044495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.061347 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:42.093888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0599b183f7fa4aa7b6bad38b25f91f34', 'request_id': 'req-0207d759-d59d-4f0c-9efe-23ab7082efcf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:42.100153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0599b183f7fa4aa7b6bad38b25f91f34', 'request_id': 'req-0207d759-d59d-4f0c-9efe-23ab7082efcf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:42.107408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0599b183f7fa4aa7b6bad38b25f91f34', 'request_id': 'req-0207d759-d59d-4f0c-9efe-23ab7082efcf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:42.118680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] The token cache is being invalidate because project 0599b183f7fa4aa7b6bad38b25f91f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:42.118970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:42.119607 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0207d759-d59d-4f0c-9efe-23ab7082efcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:42.121149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1149/2301] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:41 2022] DELETE /identity/v3/projects/0599b183f7fa4aa7b6bad38b25f91f34 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:42.125984 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.126468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.126567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.144441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:42.146294 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:42.211747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1153/2302] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:42.220743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.221032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.221304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.240983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:42.242194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:42.250864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1150/2303] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:42.255090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.266471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f570e0-57cd-4c27-b5db-4241a6c48a0f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.273232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.274341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.274749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.275102 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.276012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:42.276326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.278343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.291029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1154/2304] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:42.295359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.306566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cbc1d-31a9-4d43-9626-2a1d1fa3f887 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.311217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.311901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.312181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.312455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.313190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.315078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.342975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1151/2305] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:42.347203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.358201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230487e1-2de9-492f-86be-d3a8be124554 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.416338 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.417008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.417286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.417558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.418249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.420200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.446268 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1155/2306] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:42.450843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.461527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e36c88-9a28-494d-a70b-2af8a147f56b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.466358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.467077 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.467365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.467652 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.468531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.470172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.482394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1152/2307] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:42.486761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.499211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966058b-002b-4788-8aa1-2cf7831a5994 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.503962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.504624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.504931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.505330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.506330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.507962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94777380-b96f-4e4a-8d7b-7962f54c9891 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.518582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1156/2308] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:37:42.523015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.533697 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb1666-4148-43cf-8c50-6e923dd8a585 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.539894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.541099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.541324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.541597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] PATH_INFO: `/v3/projects/00e343d8599e47a8b3e4a7dd1eac3d52/users/16bbe85ec41e4dcc8b12efc07b65a250/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.557547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e343d8599e47a8b3e4a7dd1eac3d52, user_id=16bbe85ec41e4dcc8b12efc07b65a250, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.560219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.573681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1153/2309] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:42 2022] PUT /identity/v3/projects/00e343d8599e47a8b3e4a7dd1eac3d52/users/16bbe85ec41e4dcc8b12efc07b65a250/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:42.584046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.584432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.584763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.612185 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:42.616537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] MFA Rules not processed for user `16bbe85ec41e4dcc8b12efc07b65a250`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:42.618929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:42.690844 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1157/2310] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:42.696380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.776502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bcc3c-d98b-4d17-9057-4796e248516d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.781131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.781790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.782070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.782344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.783025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.784953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.816889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1154/2311] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/projects => generated 399 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:42.818673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.829037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6e91-f0f3-4268-a261-47a328996cbe None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.887735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.888446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.888727 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.889007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.889852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.892129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.917212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1158/2312] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:42 2022] POST /identity/v3/users => generated 306 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:42.918437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.929443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144aaedc-54ea-4f31-93da-1e3c88f1edcf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.934383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.935177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.935482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.935759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.936558 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.938203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:42.949711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1155/2313] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:42.953964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:42.964871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e82c2c-db26-4d8a-9c93-9a955b5783e3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:42.970440 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:42.971375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:42.971642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:42.971918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] PATH_INFO: `/v3/projects/eb105867987f48379d1d7ca49b5221dc/users/b2c0d97717c347e897ae09a92a79ecb7/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:42.989939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb105867987f48379d1d7ca49b5221dc, user_id=b2c0d97717c347e897ae09a92a79ecb7, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:42.995052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:43.023562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1159/2314] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:42 2022] PUT /identity/v3/projects/eb105867987f48379d1d7ca49b5221dc/users/b2c0d97717c347e897ae09a92a79ecb7/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:43.024725 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:43.036084 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6c235-0afc-4802-b490-93a9289ccd71 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:43.046550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:43.112357 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:43.114014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:43.115059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:43.115594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:43.117516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:43.119475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb05b7-6810-4aae-8dfb-f8266cf7cef8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:43.137155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1156/2315] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:43.140972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:43.156310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13db81-45ba-4831-9400-64eee2b7f05b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:43.162583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:43.163783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:43.165918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:43.166084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] PATH_INFO: `/v3/projects/eb105867987f48379d1d7ca49b5221dc/users/b2c0d97717c347e897ae09a92a79ecb7/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:43.172695 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb105867987f48379d1d7ca49b5221dc, user_id=b2c0d97717c347e897ae09a92a79ecb7, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:43.176512 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:43.192890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1160/2316] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:43 2022] PUT /identity/v3/projects/eb105867987f48379d1d7ca49b5221dc/users/b2c0d97717c347e897ae09a92a79ecb7/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:43.206414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:43.206699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:43.207098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:43.240952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] MFA Rules not processed for user `b2c0d97717c347e897ae09a92a79ecb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:43.242362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:43.365719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1157/2317] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:43.374451 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:43.376809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:43.480141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45271daa-a4b6-4a3b-b661-be6310d34cdd admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:43.486553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:43.487721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:43.488331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:43.488949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:43.518420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:43.520286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad423920-b6e1-4d97-9262-39043405c6d3 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:43.527559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:43.539899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1161/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:43.625384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:43.638176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b70265-6b31-4d7c-abbc-5f4ec46c8345 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:43.643144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:43.643822 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:43.644109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:43.644384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:43.729129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:43.730603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:43.740093 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1158/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:45.939664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1131d-ac78-4379-ad63-73978425c6f9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:45.940596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1131d-ac78-4379-ad63-73978425c6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:45.941095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1131d-ac78-4379-ad63-73978425c6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:45.963335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-d0d1131d-ac78-4379-ad63-73978425c6f9 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:45.965245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-d0d1131d-ac78-4379-ad63-73978425c6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:46.034415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1162/2320] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:37:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:46.077845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:46.079523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:46.120399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:46.153813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755e46c-6f62-4a22-bc2d-874c905ea7e8 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:46.158888 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:46.159569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:46.159853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:46.160129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:46.168847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:46.170458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:46.178948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1159/2321] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:37:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:37:46.709544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369afb-2d72-4ea2-a2cd-6833f80f6005 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:46.709831 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369afb-2d72-4ea2-a2cd-6833f80f6005 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:46.710107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc369afb-2d72-4ea2-a2cd-6833f80f6005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:46.728195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-fc369afb-2d72-4ea2-a2cd-6833f80f6005 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:46.729391 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-fc369afb-2d72-4ea2-a2cd-6833f80f6005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:46.736857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1163/2322] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:37:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:46.749360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:46.760142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76692e28-adc0-4989-8a14-b23e0be2782f service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:46.765099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:46.765765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:46.766049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:46.766323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:46.775565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:46.777143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:46.787462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1160/2323] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:37:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:37:47.012163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37f73-f75c-49a2-a3e2-8e38432fb06c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:47.012645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37f73-f75c-49a2-a3e2-8e38432fb06c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:47.012857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc37f73-f75c-49a2-a3e2-8e38432fb06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:47.031012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-bfc37f73-f75c-49a2-a3e2-8e38432fb06c None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:47.032264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bfc37f73-f75c-49a2-a3e2-8e38432fb06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:47.041613 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1164/2324] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:47.053759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:47.067976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492e2ae-e92e-455e-ad24-1ace48f75ce0 service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:47.073248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:47.073902 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:47.074188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:47.074466 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:47.084655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:47.086260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:47.095121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1161/2325] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:37:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:37:47.442188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:47.442697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:47.442986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:47.461964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:47.463155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:47.470317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1165/2326] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:47.483797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:47.494182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcb986-23f2-4faf-b7bc-f11be94dbb43 service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:47.499177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:47.499841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:47.500120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:47.500393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:47.508553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:47.510113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feff0ae-3e5b-48eb-9456-70b62cee2422 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:47.518139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1162/2327] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:37:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:37:50.638923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:50.640357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:50.717473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad71150-478d-491d-8472-5a5e18e8db5f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:50.723796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:50.724897 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:50.725227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:50.725627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] PATH_INFO: `/v3/users/35dfd2a073164dac83acbf686ad7fb8b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:50.727012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] RBAC: Authorizing `identity:delete_user(user_id=35dfd2a073164dac83acbf686ad7fb8b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:50.729104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:50.821078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35dfd2a073164dac83acbf686ad7fb8b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:50.831596 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35dfd2a073164dac83acbf686ad7fb8b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:50.846869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35dfd2a073164dac83acbf686ad7fb8b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:50.847709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35dfd2a073164dac83acbf686ad7fb8b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:50.855455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35dfd2a073164dac83acbf686ad7fb8b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:50.861741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1166/2328] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:50 2022] DELETE /identity/v3/users/35dfd2a073164dac83acbf686ad7fb8b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:50.866784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:50.867255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:50.867420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:50.885632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:50.886081 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:50.953957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1163/2329] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:50.964973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:50.966633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:50.971936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:50.982082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:50.983761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:50.995482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cee20d-3eb8-4a1e-bd49-388b5285af4a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.000848 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.001720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.001994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.002264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] PATH_INFO: `/v3/users/16bbe85ec41e4dcc8b12efc07b65a250` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.003498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16bbe85ec41e4dcc8b12efc07b65a250)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.006621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.027248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4cdc-ad93-4de2-9876-25c0b1cbd73e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.030921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.031597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.031879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.032149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.043082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.045382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f42a0b-f64c-4d66-b0b1-981d8bf857d4 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.055913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1167/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:51.061703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16bbe85ec41e4dcc8b12efc07b65a250'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.070311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16bbe85ec41e4dcc8b12efc07b65a250'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.070895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16bbe85ec41e4dcc8b12efc07b65a250'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.084616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16bbe85ec41e4dcc8b12efc07b65a250'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.089939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:51.092149 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16bbe85ec41e4dcc8b12efc07b65a250'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.101097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1164/2331] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:50 2022] DELETE /identity/v3/users/16bbe85ec41e4dcc8b12efc07b65a250 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:51.105878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.105962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.106219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.124086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:51.125337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:51.190430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1168/2332] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:51.204965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:51.206582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:51.275696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:51.277425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:51.279600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d369501-6498-4e9e-941c-72bc3117b18a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.285035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.287534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.287634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.289135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.293655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5108f-c0a0-4116-b54e-fbdffdce7b5a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.300638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.302559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.302848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.303155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] PATH_INFO: `/v3/projects/6d626a45f8a74c748b720ed295079fed` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.305138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d626a45f8a74c748b720ed295079fed)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.307086 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.307645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.309506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.323400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1165/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:37:51.374277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d626a45f8a74c748b720ed295079fed', 'request_id': 'req-a567ac5a-8f88-4d8e-8cbe-68212bce994b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.386017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d626a45f8a74c748b720ed295079fed', 'request_id': 'req-a567ac5a-8f88-4d8e-8cbe-68212bce994b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.393493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d626a45f8a74c748b720ed295079fed', 'request_id': 'req-a567ac5a-8f88-4d8e-8cbe-68212bce994b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.406396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] The token cache is being invalidate because project 6d626a45f8a74c748b720ed295079fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:51.406723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.407567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.410181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1169/2334] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:51 2022] DELETE /identity/v3/projects/6d626a45f8a74c748b720ed295079fed => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:51.684456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:51.686063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:51.699686 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752fe9f-84f9-4204-82d9-4aa1a0332bf6 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.757432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.758295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.758577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.758851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] PATH_INFO: `/v3/projects/00e343d8599e47a8b3e4a7dd1eac3d52` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.760054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e343d8599e47a8b3e4a7dd1eac3d52)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.761962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.812921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e343d8599e47a8b3e4a7dd1eac3d52', 'request_id': 'req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.818960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e343d8599e47a8b3e4a7dd1eac3d52', 'request_id': 'req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.826390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e343d8599e47a8b3e4a7dd1eac3d52', 'request_id': 'req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.839788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] The token cache is being invalidate because project 00e343d8599e47a8b3e4a7dd1eac3d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:51.840080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.840734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.842499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1166/2335] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:51 2022] DELETE /identity/v3/projects/00e343d8599e47a8b3e4a7dd1eac3d52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:51.846459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:51.848070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:51.861967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567ac5a-8f88-4d8e-8cbe-68212bce994b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:51.887952 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:51.930367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:51.931459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:51.931742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:51.932017 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] PATH_INFO: `/v3/users/b2c0d97717c347e897ae09a92a79ecb7` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:51.933290 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c0d97717c347e897ae09a92a79ecb7)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:51.935206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:51.984269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c0d97717c347e897ae09a92a79ecb7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:51.991840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c0d97717c347e897ae09a92a79ecb7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.003771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c0d97717c347e897ae09a92a79ecb7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.004429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c0d97717c347e897ae09a92a79ecb7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.011004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-8640c486-351f-40c7-a5f3-b9be8964d623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c0d97717c347e897ae09a92a79ecb7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.016782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1170/2336] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:37:51 2022] DELETE /identity/v3/users/b2c0d97717c347e897ae09a92a79ecb7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:52.481993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.483634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:52.497523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e2afa-3911-40ae-a6f4-fe97416d7d9d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.525219 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:52.557647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.558443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.558728 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.559355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] PATH_INFO: `/v3/projects/eb105867987f48379d1d7ca49b5221dc` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.560224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb105867987f48379d1d7ca49b5221dc)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:52.562140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:52.615696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb105867987f48379d1d7ca49b5221dc', 'request_id': 'req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.620963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb105867987f48379d1d7ca49b5221dc', 'request_id': 'req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.630591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb105867987f48379d1d7ca49b5221dc', 'request_id': 'req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.641518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] The token cache is being invalidate because project eb105867987f48379d1d7ca49b5221dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:37:52.641803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.642419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-8fd65fd0-a849-4157-bf9c-4e52a9869c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:37:52.643991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1167/2337] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:37:52 2022] DELETE /identity/v3/projects/eb105867987f48379d1d7ca49b5221dc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:37:52.649062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.649357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.649643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.668078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:52.669267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:52.727242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1171/2338] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:52.735067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.735387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.735672 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.754253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:52.756146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:52.764289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1168/2339] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:52.768387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.777896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12984e-6300-4e01-8cec-266f3daefc71 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.782494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.783151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.783468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.783747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.784469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:37:52.784775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:52.786456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:52.798018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1172/2340] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:37:52.802743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.812109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c978-e9b1-4178-a4ed-e8d8d582954b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.817425 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.818073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.818359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.818641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.819373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:52.821285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:52.847530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1169/2341] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:37:52 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:37:52.851717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.861545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a75ab-d550-481f-bbad-f37a521f2e6e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.893794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:52.921181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.921848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.922127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.922395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.923071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:52.925105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:52.944625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1173/2342] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:37:52 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:37:52.948779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.958907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc6772-1414-415f-96bc-1b4115eccc54 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.963486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.964132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.964416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.964691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:52.965411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:52.966997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:52.977715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1170/2343] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:52.981566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:52.993236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14398-3183-4ad9-9236-36133ffba132 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:52.998139 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:52.998809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:52.999091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:52.999407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:53.000141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:53.001821 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737b42f-a313-4838-b419-e5b508b141e7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:53.012506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1174/2344] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:37:53.016525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:53.027417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899c07b-2a15-463d-8448-768bacdd2bcc admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:53.032204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:53.033114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:53.033396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:53.033671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] PATH_INFO: `/v3/projects/0e1f5e20c628444587bc0a45ef0cb76c/users/e005e71a8ec345478de3f11af3a3133b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:53.049198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1f5e20c628444587bc0a45ef0cb76c, user_id=e005e71a8ec345478de3f11af3a3133b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:53.052432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:53.065379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1171/2345] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:37:53 2022] PUT /identity/v3/projects/0e1f5e20c628444587bc0a45ef0cb76c/users/e005e71a8ec345478de3f11af3a3133b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:37:53.075571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c8d7-305b-4fdf-b0fc-dbe24f26d0b4 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:53.075870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c8d7-305b-4fdf-b0fc-dbe24f26d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:53.076154 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c8d7-305b-4fdf-b0fc-dbe24f26d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:53.104467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ef22c8d7-305b-4fdf-b0fc-dbe24f26d0b4 None None] MFA Rules not processed for user `e005e71a8ec345478de3f11af3a3133b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:37:53.105694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ef22c8d7-305b-4fdf-b0fc-dbe24f26d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:53.174674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1175/2346] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:37:53.186393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:37:53.188120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:37:53.244504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37fd16-7e56-48af-9749-dc982eb9d5a4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:37:53.248217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:37:53.248862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:37:53.249146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:37:53.249421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:37:53.257718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:37:53.259137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:37:53.262622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:37:53.272864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1172/2347] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:37:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:02.485668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d149f2-6ab3-4e24-bd50-d621b30dd32b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:02.486167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d149f2-6ab3-4e24-bd50-d621b30dd32b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:02.486261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d149f2-6ab3-4e24-bd50-d621b30dd32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:02.505992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-67d149f2-6ab3-4e24-bd50-d621b30dd32b None None] MFA Rules not processed for user `7ed56ed8be784927917986375a76df4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:02.507412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-67d149f2-6ab3-4e24-bd50-d621b30dd32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:02.577370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1176/2348] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:03.275003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:07.073472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:07.075062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:07.145801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073c2cf-ff5b-4035-8e51-2369fa461871 service glance] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:07.150380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:07.151301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:07.151588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:07.151860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] PATH_INFO: `/v3/users/b1158edcff7747e19e369a0d01227798` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:07.153046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1158edcff7747e19e369a0d01227798)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:07.154954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:07.214097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1158edcff7747e19e369a0d01227798'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.224751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1158edcff7747e19e369a0d01227798'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.225477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1158edcff7747e19e369a0d01227798'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.242569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1158edcff7747e19e369a0d01227798'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.249757 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1158edcff7747e19e369a0d01227798'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.256010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1173/2349] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:38:07 2022] DELETE /identity/v3/users/b1158edcff7747e19e369a0d01227798 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:07.260659 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f915b-3918-4322-920c-125354688a38 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:07.261112 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f915b-3918-4322-920c-125354688a38 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:07.261403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f915b-3918-4322-920c-125354688a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:07.279985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-276f915b-3918-4322-920c-125354688a38 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:07.281295 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-276f915b-3918-4322-920c-125354688a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:07.330468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:07.349236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1177/2350] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:07.358764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:07.360502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:07.420420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479065a-c513-4138-9e39-b5ebbc64e6cc admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:07.424704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:07.425356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:07.425640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:07.425962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:07.442761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:07.444524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fa34e-34c2-47b1-86b6-ee4c00bfe433 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:07.456693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1174/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:38:07.842944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f915b-3918-4322-920c-125354688a38 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:07.844643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-276f915b-3918-4322-920c-125354688a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:07.864101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f915b-3918-4322-920c-125354688a38 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:07.869386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:07.870265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:07.870544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:07.870909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] PATH_INFO: `/v3/projects/f4e4dfcce8754d97973892db79a48859` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:07.871994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e4dfcce8754d97973892db79a48859)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:07.873907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:07.919724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e4dfcce8754d97973892db79a48859', 'request_id': 'req-1a095560-4e1a-476d-90b2-6d2c74aec879'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.928479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e4dfcce8754d97973892db79a48859', 'request_id': 'req-1a095560-4e1a-476d-90b2-6d2c74aec879'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.933975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e4dfcce8754d97973892db79a48859', 'request_id': 'req-1a095560-4e1a-476d-90b2-6d2c74aec879'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.945524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] The token cache is being invalidate because project f4e4dfcce8754d97973892db79a48859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:38:07.945809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.946430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-1a095560-4e1a-476d-90b2-6d2c74aec879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:07.947996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1178/2352] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:38:07 2022] DELETE /identity/v3/projects/f4e4dfcce8754d97973892db79a48859 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:07.952963 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:07.953326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:07.953679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:07.971226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:07.972527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:07.989479 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:08.032907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1175/2353] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:38:08.041846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.042144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.042429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.061342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:08.062534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:08.071068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1179/2354] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:08.075311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.085068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e390f-ccd0-4e40-8443-b27b37fbbf01 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.089555 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.090211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.090491 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.090761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.091505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:38:08.091809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.093373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.104681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1176/2355] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:38:08.108961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.118770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf35116-8822-4b10-aaaa-a5b2f403141f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.123367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.124016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.124296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.124571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.125249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.127136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.153823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1180/2356] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:38:08 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:38:08.157985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.168354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0dc51-6ad6-4c6f-a975-34dcd83d9039 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.225751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.226427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.226714 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.226994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.227739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.229666 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.249461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1177/2357] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:38:08 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:38:08.253574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.264178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13ec24-6f7a-4bfc-87c6-13d9e23e5145 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.268734 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.269386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.270280 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.270563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.271313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.272907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.283396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1181/2358] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:38:08.287492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.297523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b9d3c-4029-49eb-b2e0-2785f16a9900 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.302127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.302803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.303079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.303391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.304110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.305678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca7193-1754-4fa8-8126-6a8b298056d9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.315370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1178/2359] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:38:08.319878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.329714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e8b7c-cdc6-47ef-824c-7e5589bd2926 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.335027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.335936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.336218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.336492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] PATH_INFO: `/v3/projects/6227867f9e0347d3ac9b61013cdcce4a/users/9dc77e752e8d42b89882120329d69ac4/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.350567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6227867f9e0347d3ac9b61013cdcce4a, user_id=9dc77e752e8d42b89882120329d69ac4, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.353270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.365927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1182/2360] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:38:08 2022] PUT /identity/v3/projects/6227867f9e0347d3ac9b61013cdcce4a/users/9dc77e752e8d42b89882120329d69ac4/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:08.376106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.376395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.376671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.405912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] MFA Rules not processed for user `9dc77e752e8d42b89882120329d69ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:08.407121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:08.469706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1179/2361] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:08.481010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:08.482687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:08.538629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1763dc-e992-43eb-92a9-69307256dd16 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:08.542439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:08.543068 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:08.543376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:08.543654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:08.552671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:08.554135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b9a2a-b340-47e4-8342-78b823cf363b service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:08.562912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1183/2362] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:13.985662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:13.987006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.009878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:14.059275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4766da1-63dd-441f-856b-89de2f42e419 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:14.063816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:14.064699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.064986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.065260 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] PATH_INFO: `/v3/users/e005e71a8ec345478de3f11af3a3133b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.066747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e005e71a8ec345478de3f11af3a3133b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:14.069346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:14.122261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e005e71a8ec345478de3f11af3a3133b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.132192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e005e71a8ec345478de3f11af3a3133b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.132882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e005e71a8ec345478de3f11af3a3133b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.147729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e005e71a8ec345478de3f11af3a3133b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.155233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e005e71a8ec345478de3f11af3a3133b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.161968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1180/2363] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:38:13 2022] DELETE /identity/v3/users/e005e71a8ec345478de3f11af3a3133b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:38:14.167093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.167625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.167722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.185986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:14.187274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.193111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:14.250781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1184/2364] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:14.260665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:14.263937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.339043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f037d-d00c-4a4c-9cf8-58878e720947 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:14.342906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:14.343767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.344057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.344417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.355560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:14.357134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b6f2a5-045d-4423-9372-9aed082b0dac service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:14.384996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1181/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:14.704364 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:14.705858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.723551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e1d8a-cc9d-4294-be51-16a5d1016be2 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:14.728298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:14.729169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.729447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.729827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] PATH_INFO: `/v3/projects/0e1f5e20c628444587bc0a45ef0cb76c` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.730937 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1f5e20c628444587bc0a45ef0cb76c)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:14.732963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:14.782504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1f5e20c628444587bc0a45ef0cb76c', 'request_id': 'req-45334b7a-2fb6-464f-a057-255b60218cb1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.791623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1f5e20c628444587bc0a45ef0cb76c', 'request_id': 'req-45334b7a-2fb6-464f-a057-255b60218cb1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.797619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1f5e20c628444587bc0a45ef0cb76c', 'request_id': 'req-45334b7a-2fb6-464f-a057-255b60218cb1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.809246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] The token cache is being invalidate because project 0e1f5e20c628444587bc0a45ef0cb76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:38:14.809534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.810179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-45334b7a-2fb6-464f-a057-255b60218cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:14.811754 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1185/2366] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:38:14 2022] DELETE /identity/v3/projects/0e1f5e20c628444587bc0a45ef0cb76c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:14.819276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.821858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.822133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.841433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:14.842643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.877483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:14.907071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1182/2367] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:14.916460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.916751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.917032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.940468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:14.942487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:14.953969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1186/2368] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:14.955216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:14.965538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f379c-f959-403e-ab85-1a9d33937957 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:14.970168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:14.970817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:14.971096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:14.971405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:14.972236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:38:14.972560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:14.974130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:14.987328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1183/2369] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:38:14.990532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.000105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0bc05-7264-4b19-8698-3a2b934515ab None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.005752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.006410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.006691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.006961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.007778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.009690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.038371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1187/2370] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:38:14 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:38:15.043032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.055419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b2c1-bb2f-410f-abf5-7604ebda1da0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.127918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.129301 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.129741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.130024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.130743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.133021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.167577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1184/2371] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:38:15 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:38:15.171823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.184597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ce31a-78c5-4c66-8c25-3dfa8216c3f1 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.192126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.193422 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.193787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.194113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.195040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.197649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.213629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1188/2372] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:38:15.217576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.228143 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f780cf6-fd17-4008-a83a-e18c94060bd6 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.233156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.233895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.234258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.234435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.235171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.243708 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7ff8d-6f99-4ce0-83ec-3c5d16486cb6 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.258326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1185/2373] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:38:15.259482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.270244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd202bb0-7da2-420c-9cdc-5c1746dd1ac4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.277096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.278079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.278359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.278631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] PATH_INFO: `/v3/projects/d80d950588ad45939ec7478272a163e5/users/5404c7f494dd4835af66d6c571cfb04c/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.294715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80d950588ad45939ec7478272a163e5, user_id=5404c7f494dd4835af66d6c571cfb04c, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.297449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.310384 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1189/2374] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:38:15 2022] PUT /identity/v3/projects/d80d950588ad45939ec7478272a163e5/users/5404c7f494dd4835af66d6c571cfb04c/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:15.321613 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b17f0-3131-4fd5-a382-11c8b72c3773 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.321906 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b17f0-3131-4fd5-a382-11c8b72c3773 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.322187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b17f0-3131-4fd5-a382-11c8b72c3773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.348381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1c1b17f0-3131-4fd5-a382-11c8b72c3773 None None] MFA Rules not processed for user `5404c7f494dd4835af66d6c571cfb04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:15.349625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1c1b17f0-3131-4fd5-a382-11c8b72c3773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:15.413830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1186/2375] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:15.429235 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:15.430428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:15.468774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:15.495116 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdfe91-faa1-4da0-9e0d-c31ea16795e5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:15.498922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:15.499711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:15.500047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:15.500346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:15.510340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:15.512232 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:15.521961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1190/2376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:19.878986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b897-4d86-477c-839f-22d5a6c3a2df None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:19.879510 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b897-4d86-477c-839f-22d5a6c3a2df None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:19.879616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902b897-4d86-477c-839f-22d5a6c3a2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:19.897445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f902b897-4d86-477c-839f-22d5a6c3a2df None None] MFA Rules not processed for user `bc74ebac003a4f828fdb44db6dc42887`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:19.898649 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f902b897-4d86-477c-839f-22d5a6c3a2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:19.931134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:19.960348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1187/2377] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:25.517500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=12 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:29.951982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=9 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:31.257052 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:31.258449 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:31.325526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a67c7-91ab-4e54-8086-569fb5e5eb35 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:31.330289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:31.331167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:31.331486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:31.331764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] PATH_INFO: `/v3/users/9dc77e752e8d42b89882120329d69ac4` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:31.333006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc77e752e8d42b89882120329d69ac4)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:31.334956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:31.388481 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc77e752e8d42b89882120329d69ac4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:31.395955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc77e752e8d42b89882120329d69ac4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:31.408040 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc77e752e8d42b89882120329d69ac4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:31.408778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc77e752e8d42b89882120329d69ac4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:31.416247 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc77e752e8d42b89882120329d69ac4'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:31.422234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1191/2378] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:38:31 2022] DELETE /identity/v3/users/9dc77e752e8d42b89882120329d69ac4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:31.428390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:31.428891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:31.428984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:31.451131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:31.452381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:31.513802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1188/2379] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:31.523355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:31.524748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:31.581516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6cfe8-5551-498f-8cad-4dfa0c644d72 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:31.585219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:31.585867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:31.586150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:31.586428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:31.595395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:31.597465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:31.606876 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1192/2380] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:31.964413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:31.966293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:31.986464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dca881-acc8-417a-9a70-3ed484d2c430 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:31.992571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:31.993719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:31.994129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:31.994329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] PATH_INFO: `/v3/projects/6227867f9e0347d3ac9b61013cdcce4a` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:31.995520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6227867f9e0347d3ac9b61013cdcce4a)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:31.997427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:32.048923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6227867f9e0347d3ac9b61013cdcce4a', 'request_id': 'req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:32.056324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6227867f9e0347d3ac9b61013cdcce4a', 'request_id': 'req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:32.064682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6227867f9e0347d3ac9b61013cdcce4a', 'request_id': 'req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:32.075813 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] The token cache is being invalidate because project 6227867f9e0347d3ac9b61013cdcce4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:38:32.076094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:32.076754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d4a4cb27-2945-40dc-ba91-cfb83c9a326e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:38:32.078316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1189/2381] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:38:31 2022] DELETE /identity/v3/projects/6227867f9e0347d3ac9b61013cdcce4a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:38:41.602665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:42.072739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:38:49.075039 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:49.076498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:49.151725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4556b3-ee6f-4ea0-964e-5deff4b52e22 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:49.212289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:49.212962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:49.213251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:49.213531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:49.215403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:49.227255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:49.228891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:49.239307 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1193/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:38:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:38:56.305062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:56.305561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:56.305664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:56.323586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:38:56.324803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:56.384749 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1190/2383] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:38:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:38:56.443227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:38:56.445054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:38:56.522608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77368ad-4e9c-467d-ab2f-de01ff74d820 service placement] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:38:56.530209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:38:56.530870 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:38:56.531172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:38:56.531472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:38:56.541100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:38:56.542704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d713f-5e94-4ce8-a09a-2ccc41194168 service swift] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:38:56.552265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1194/2384] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:38:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:39:06.377283 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=25 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:06.548076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:08.802682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:08.804350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:08.873234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d8579-17d3-41cc-be62-15c544ba9c31 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:08.878594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:08.879499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:08.879779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:08.880048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] PATH_INFO: `/v3/users/5404c7f494dd4835af66d6c571cfb04c` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:08.881271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5404c7f494dd4835af66d6c571cfb04c)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:08.883213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:08.931941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5404c7f494dd4835af66d6c571cfb04c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:08.940418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5404c7f494dd4835af66d6c571cfb04c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:08.941050 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5404c7f494dd4835af66d6c571cfb04c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:08.954122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5404c7f494dd4835af66d6c571cfb04c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:08.961067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5404c7f494dd4835af66d6c571cfb04c'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:08.967399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1191/2385] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:39:08 2022] DELETE /identity/v3/users/5404c7f494dd4835af66d6c571cfb04c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:39:08.972037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:08.972578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:08.972786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:08.990524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:08.991788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:09.052221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1195/2386] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:09.061802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:09.063894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:09.124681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961f286-b74e-41fd-8a5d-7d38be55f29f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:09.129153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:09.129816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:09.130098 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:09.130454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:09.140259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:09.141961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d2d6-73d9-44f6-b8cb-696e93898582 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:09.151779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1192/2387] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:09.426156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:09.427961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:09.449678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0e711-ddbd-469b-a7bf-ed262dea7278 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:09.458627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:09.459736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:09.460184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:09.460525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] PATH_INFO: `/v3/projects/d80d950588ad45939ec7478272a163e5` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:09.461869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80d950588ad45939ec7478272a163e5)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:09.463914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:09.514453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80d950588ad45939ec7478272a163e5', 'request_id': 'req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:09.522603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80d950588ad45939ec7478272a163e5', 'request_id': 'req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:09.529409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80d950588ad45939ec7478272a163e5', 'request_id': 'req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:09.541330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] The token cache is being invalidate because project d80d950588ad45939ec7478272a163e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:39:09.541624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:09.542305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-b490d8e9-eccd-4a2e-b8cd-83e7fe0f0a58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:09.543889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1196/2388] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:39:09 2022] DELETE /identity/v3/projects/d80d950588ad45939ec7478272a163e5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:39:11.749651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:11.750254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:11.750254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:11.751784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1193/2389] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:39:11.760813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47176-28f3-4db9-8323-bca694294cb2 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:11.761289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47176-28f3-4db9-8323-bca694294cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:11.761505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47176-28f3-4db9-8323-bca694294cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:11.780541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4ab47176-28f3-4db9-8323-bca694294cb2 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:11.781741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4ab47176-28f3-4db9-8323-bca694294cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:11.854344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1197/2390] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:39:11.868325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:11.870078 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:11.899791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:11.939186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da877fe-7099-41a9-858e-9735dc7e4e25 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:11.952512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:11.953208 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:11.953587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:11.953834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:11.965647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:11.967007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f37030-6603-4e53-9f03-03271d476f08 service glance] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:11.980289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1194/2391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:39:12.051035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.051360 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.051645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.053153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1198/2392] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:39:12.058256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbcc9-a4d8-4962-be5c-28dda5fc3a2a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.058548 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbcc9-a4d8-4962-be5c-28dda5fc3a2a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.058986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbcc9-a4d8-4962-be5c-28dda5fc3a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.076570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e7bcbcc9-a4d8-4962-be5c-28dda5fc3a2a None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:12.077761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e7bcbcc9-a4d8-4962-be5c-28dda5fc3a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:12.133942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1195/2393] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:12.146838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:12.148663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:12.163148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22067b-31b7-4c44-9a1d-cfcd6631ed05 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:12.166750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:12.167429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.167715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.167992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.175848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:12.183495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea05901-f138-428e-b33d-b8c76a28faf6 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:12.194690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1199/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:12.317153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416852-33b4-47f4-8b5d-3758f0a515fc None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.317446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416852-33b4-47f4-8b5d-3758f0a515fc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.317751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416852-33b4-47f4-8b5d-3758f0a515fc None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.319192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1196/2395] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:39:12.324465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca16d-18ab-494e-8c97-d6925c616bf1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.324773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca16d-18ab-494e-8c97-d6925c616bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.325050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dca16d-18ab-494e-8c97-d6925c616bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.332009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:12.346421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-58dca16d-18ab-494e-8c97-d6925c616bf1 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:12.347669 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-58dca16d-18ab-494e-8c97-d6925c616bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:12.356980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1200/2396] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:39:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:12.367569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6416852-33b4-47f4-8b5d-3758f0a515fc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:12.378707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6416852-33b4-47f4-8b5d-3758f0a515fc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:12.382433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:12.383071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:12.383380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:12.383658 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:12.394381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:12.395895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:12.406782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1197/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:14.813761 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6116828-899b-4439-96b2-f61a0a760292 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:14.814243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6116828-899b-4439-96b2-f61a0a760292 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:14.814243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6116828-899b-4439-96b2-f61a0a760292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:14.832058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a6116828-899b-4439-96b2-f61a0a760292 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:14.833270 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a6116828-899b-4439-96b2-f61a0a760292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:14.888902 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1201/2398] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:39:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:14.901692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:14.903063 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:14.972501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7327aa-9c0c-48f3-b9cf-bd3c2aa90570 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:14.977570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:14.978231 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:14.978518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:14.978798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:14.987049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:14.988687 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952275bc-3be1-4854-b4e2-92a7f5cc8e34 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:14.996863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1198/2399] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:39:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:39:15.029808 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.030183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.030469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.032149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1202/2400] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:39:15.057576 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.058169 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.058605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.081271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] MFA Rules not processed for user `9dddb4f1f2b443049a249f9ab3eeb23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:15.082981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:15.119491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:15.121535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:15.181698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:15.225412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1199/2401] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:39:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:15.241173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:15.242198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:15.246252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf825624-7aee-4fba-b9ca-65be419c5e81 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:15.253159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:15.254302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.254699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.255083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] PATH_INFO: `/v3/users/7ed56ed8be784927917986375a76df4c` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.256617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed56ed8be784927917986375a76df4c)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:15.258947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:15.272238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b0e9f-25b8-4ca9-a1bb-290434bf6242 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:15.292639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:15.293836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.294275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.294980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.324669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:15.326584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:15.343442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1200/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:15.350539 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed56ed8be784927917986375a76df4c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.361508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed56ed8be784927917986375a76df4c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.401355 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed56ed8be784927917986375a76df4c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.403304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed56ed8be784927917986375a76df4c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.420061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-97b526a6-b1a7-43f5-8948-0b5439385322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed56ed8be784927917986375a76df4c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.437656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1203/2403] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:39:15 2022] DELETE /identity/v3/users/7ed56ed8be784927917986375a76df4c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:39:15.708140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:15.727618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfa61c-5f93-4df1-9ce6-d21d907f0c1b service placement] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:15.801664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:15.802638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.802928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.803204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] PATH_INFO: `/v3/projects/298d7611d92a47fd94dd66c8ff10a7f3` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.804718 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298d7611d92a47fd94dd66c8ff10a7f3)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:15.808990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:15.867500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298d7611d92a47fd94dd66c8ff10a7f3', 'request_id': 'req-0c733034-9113-4a55-9f88-707327a6c413'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.873344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298d7611d92a47fd94dd66c8ff10a7f3', 'request_id': 'req-0c733034-9113-4a55-9f88-707327a6c413'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.884229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298d7611d92a47fd94dd66c8ff10a7f3', 'request_id': 'req-0c733034-9113-4a55-9f88-707327a6c413'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.898792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] The token cache is being invalidate because project 298d7611d92a47fd94dd66c8ff10a7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:39:15.899207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.899898 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0c733034-9113-4a55-9f88-707327a6c413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:39:15.901701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1201/2404] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:39:15 2022] DELETE /identity/v3/projects/298d7611d92a47fd94dd66c8ff10a7f3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:39:15.908968 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:15.909500 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:15.909598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:15.929609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:15.930841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:15.947310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:16.006651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1204/2405] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:16.014217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.014506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.014772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.033659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:16.034878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:16.043326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1202/2406] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:16.048132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.059050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1c3b3-9181-412e-ba8f-9b3deeeac7f6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.064979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.065652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.065933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.066205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.066934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:39:16.067239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.068874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.081725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1205/2407] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:39:16.086023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.096011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7047235-3300-4d90-84ab-cb13ffc9ade1 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.108857 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.109539 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.110556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.110844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.111838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.113991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.143402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1203/2408] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:39:16 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:39:16.146267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.157292 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf04b9-9d5e-4798-9c99-bb0f531540ff admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.221693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.222687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.223134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.223641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.224642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.227145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.250839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1206/2409] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:39:16 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:39:16.255346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.267447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40206d-a87e-41be-9fb5-3361a7d2f7d5 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.272417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.273211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.273497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.273773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.274659 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.276513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.291117 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1204/2410] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:39:16.295123 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.306251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d46e2-a063-405c-80cf-8cf4a759eecb admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.311271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.311975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.312256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.312528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.313403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.315024 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbce950-180f-4fed-b37e-ab1bcd6a9ed8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.325905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1207/2411] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:39:16.330167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.340356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5a078-5403-4f09-9082-9c6f2228fcc2 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.345767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.346662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.346942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.347215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] PATH_INFO: `/v3/projects/65ccb95f08764f99b5a5c1166c42410b/users/e4f9820744874f9c979cb18cbf624fb4/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.363389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ccb95f08764f99b5a5c1166c42410b, user_id=e4f9820744874f9c979cb18cbf624fb4, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.366040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.376094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:16.381592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1205/2412] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:39:16 2022] PUT /identity/v3/projects/65ccb95f08764f99b5a5c1166c42410b/users/e4f9820744874f9c979cb18cbf624fb4/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:39:16.387584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066cb9b-ff81-4afc-a855-995fbc20dfc6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.388078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066cb9b-ff81-4afc-a855-995fbc20dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.388431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066cb9b-ff81-4afc-a855-995fbc20dfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.411399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-8066cb9b-ff81-4afc-a855-995fbc20dfc6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:16.412680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8066cb9b-ff81-4afc-a855-995fbc20dfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:16.485580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1208/2413] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:16.494992 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:16.504374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:16.564622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcd661-bdf2-41b4-bb23-fad14f74b42d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:16.569685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:16.570439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:16.570803 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:16.571064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:16.585363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:16.586954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:16.603019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1206/2414] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:22.567781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:22.568400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:22.568400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:22.616049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] MFA Rules not processed for user `e4f9820744874f9c979cb18cbf624fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:22.617625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:22.641302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:22.733127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1209/2415] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:22.733795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:22.745431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519b69e-b4de-4adf-869c-49982a16510d service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:22.759756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:22.760445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:22.760726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:22.760997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:22.761703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:22.764207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:22.797167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1207/2416] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:39:22 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:39:22.801537 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:22.813256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f8a34-c90d-49e9-9be9-66efcadb6b7c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:22.880468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:22.881248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:22.881576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:22.881846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:22.882607 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:22.884624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:22.909943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1210/2417] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:39:22 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:39:22.914137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:22.925432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06fd63-b163-444a-aeb6-e869dbe01dc4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:22.930344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:22.931181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:22.931535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:22.931817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:22.932552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:22.934161 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:22.947758 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1208/2418] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:39:22.952342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:22.963972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189524-5eec-4403-8f7a-0653b2450784 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:22.970334 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:22.971244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:22.971571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:22.971843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] PATH_INFO: `/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96/users/a2e1c0d758534efa8c04869bc54ee2aa/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:22.990887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e77b1ed93048a8a6bfda9c5976ae96, user_id=a2e1c0d758534efa8c04869bc54ee2aa, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:22.994319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:23.014303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1211/2419] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:39:22 2022] PUT /identity/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96/users/a2e1c0d758534efa8c04869bc54ee2aa/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:39:23.018919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:23.030483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92f28-9d59-4715-a9d8-94e26d9324ee admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:23.120741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:23.121434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:23.121713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:23.121983 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:23.122707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:23.124345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f77a35-8469-447c-9fe3-9322b2c3eb41 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:23.139237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1209/2420] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:39:23.140363 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:23.152563 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15467ecd-d1f5-44ed-93b4-5a1c932dcb92 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:23.157343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:23.158231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:23.158505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:23.159000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] PATH_INFO: `/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96/users/a2e1c0d758534efa8c04869bc54ee2aa/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:23.161206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e77b1ed93048a8a6bfda9c5976ae96, user_id=a2e1c0d758534efa8c04869bc54ee2aa, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:23.164151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:23.175998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1212/2421] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:39:23 2022] PUT /identity/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96/users/a2e1c0d758534efa8c04869bc54ee2aa/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:39:27.880152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a2c6f-f407-4ca4-8306-d35f7e0d98ba None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:27.880747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a2c6f-f407-4ca4-8306-d35f7e0d98ba None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:27.880747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a2c6f-f407-4ca4-8306-d35f7e0d98ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:27.917463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-769a2c6f-f407-4ca4-8306-d35f7e0d98ba None None] MFA Rules not processed for user `a2e1c0d758534efa8c04869bc54ee2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:27.919593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-769a2c6f-f407-4ca4-8306-d35f7e0d98ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:27.980910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:28.010774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1210/2422] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:39:28.024195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:28.026277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:28.096742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170bbab-1b05-49f2-9364-ee6cf5b6760c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:28.100704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:28.101460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:28.101815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:28.102161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:28.129146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:28.205561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:28.207203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:28.222484 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1213/2423] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:39:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:39:28.330459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81597-c1b2-4d66-a179-9b716bf97fa0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:28.330744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81597-c1b2-4d66-a179-9b716bf97fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:28.331054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81597-c1b2-4d66-a179-9b716bf97fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:28.350051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d0f81597-c1b2-4d66-a179-9b716bf97fa0 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:28.351278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d0f81597-c1b2-4d66-a179-9b716bf97fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:28.411092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1211/2424] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:28.424499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:28.426099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:28.505128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213eaeee-6bf8-4df2-8ee4-bbc29931ed46 service glance] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:28.512527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:28.513293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:28.513652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:28.513995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:28.524236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:28.525927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:28.536105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1214/2425] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:39:35.136246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f31-df3f-47c7-894f-474d7cf70f1e None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:35.137366 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f31-df3f-47c7-894f-474d7cf70f1e None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:35.138188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc8f31-df3f-47c7-894f-474d7cf70f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:35.171147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-9fcc8f31-df3f-47c7-894f-474d7cf70f1e None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:35.173343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9fcc8f31-df3f-47c7-894f-474d7cf70f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:35.188748 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1212/2426] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:39:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:35.197569 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:35.212975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ab4d3-fd61-48b9-92cb-17b2310baaa1 service swift] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:35.225383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:35.226049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:35.226333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:35.226636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:35.244792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:35.246603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c882c4-234a-4982-887d-0b07a8fcb2e6 service swift] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:35.256053 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1215/2427] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:39:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:39:38.220100 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:38.220595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:38.220694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:38.238187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] MFA Rules not processed for user `e4f9820744874f9c979cb18cbf624fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:38.239405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:38.247608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1213/2428] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:39:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:40.694606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:40.695157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:40.695599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:40.719804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:39:40.721142 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:39:40.728886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1216/2429] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:39:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:39:40.742785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:39:40.755689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc4403-c268-4964-98c8-b1db55458136 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:39:40.761360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:39:40.762007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:39:40.762289 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:39:40.762581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:39:40.771586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:39:40.773159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3fa1b-cba1-4550-aa3a-9de25d28b5a9 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:39:40.782436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1214/2430] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:39:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 09:39:50.712356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=58 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:39:50.778744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=41 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:40:07.094048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:40:07.151503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99453a-6350-46e3-ba3b-776f71d0b935 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:40:07.161041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:40:07.161732 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:07.162013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:07.162553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:07.172520 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:40:07.173970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a88128-9b24-41b5-86c7-16d3be207243 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:40:07.184975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1217/2431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:40:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:40:17.038663 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:17.039191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:17.039191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:17.057379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] MFA Rules not processed for user `a37198e6d4424a51800655d3bd0a3ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:40:17.058705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:17.116620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1215/2432] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:40:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:40:17.179789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=26 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:40:24.373311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ed11-30e3-4707-9136-262f7d9b0209 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:24.373994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ed11-30e3-4707-9136-262f7d9b0209 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:24.374095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ed11-30e3-4707-9136-262f7d9b0209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:24.391260 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3cb2ed11-30e3-4707-9136-262f7d9b0209 None None] MFA Rules not processed for user `bc74ebac003a4f828fdb44db6dc42887`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:40:24.392473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3cb2ed11-30e3-4707-9136-262f7d9b0209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:24.447328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1218/2433] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:40:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:40:27.109527 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=25 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:40:34.440778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=25 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:40:47.333961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:40:47.335485 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:47.347711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79dc4e-d39b-4b42-a874-c747b513f060 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:40:47.351679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:40:47.352328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:47.352629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:47.352966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:47.354676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:47.416151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:40:47.417590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:40:47.427354 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1216/2434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:40:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:40:48.219243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:48.219787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:48.219987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:48.239135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] MFA Rules not processed for user `ee885a29e6fa4b1980d7ae1d03b2b476`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:40:48.240375 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:48.247882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1219/2435] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:40:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:40:56.979953 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:40:56.988604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33089f2-d098-4611-af12-db4cc85a6f7b service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:40:56.992213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:40:56.992847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:40:56.993130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:40:56.993411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:40:56.995325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:40:57.076195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:40:57.077626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:40:57.088492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1217/2436] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:40:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:40:58.232889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:07.083606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=62 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:07.232038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:07.233479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:07.287989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99b539-cda1-4d75-9cea-71571c2ad93b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:07.291525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:07.292163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:07.292444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:07.292716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:07.295158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:07.307275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:07.309119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:07.319705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1220/2437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:10.046979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:10.055668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c87d-a47e-4b52-82bc-c4829edbde27 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:10.060011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:10.060651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:10.060936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:10.061216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:10.062960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:10.076246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:10.077665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:10.088413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1218/2438] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:14.515663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:14.517149 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:14.535136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc1ea9-98c7-438a-90ae-a46248f1004b service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:14.541298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:14.541954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:14.542236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:14.542525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:14.545125 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:14.556650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:14.558306 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:14.571524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1221/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:20.083599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:24.565992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=32 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:27.299121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:27.308502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3257f-75e9-4086-9591-ebe90726bb4c service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:27.312756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:27.313433 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:27.313693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:27.313971 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:27.315941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:27.379311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:27.380772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:27.391048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1219/2440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:30.798722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:30.800258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:30.820377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7eba09-55b3-41eb-a76a-c0401cb339da service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:30.827158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:30.828082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:30.828362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:30.828632 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] PATH_INFO: `/v3/users/bc74ebac003a4f828fdb44db6dc42887` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:30.830108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc74ebac003a4f828fdb44db6dc42887)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:30.832062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:30.878753 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc74ebac003a4f828fdb44db6dc42887'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:30.886047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc74ebac003a4f828fdb44db6dc42887'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:30.897912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc74ebac003a4f828fdb44db6dc42887'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:30.898544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc74ebac003a4f828fdb44db6dc42887'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:30.905776 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a468c5b3-fd61-40ff-93db-b9e918c5fff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc74ebac003a4f828fdb44db6dc42887'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:30.911612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1222/2441] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:30 2022] DELETE /identity/v3/users/bc74ebac003a4f828fdb44db6dc42887 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:31.082579 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.092664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4bab-5978-4888-b48c-87d8a5f73759 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.157250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.158138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.158441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.158719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] PATH_INFO: `/v3/projects/13b4839566df4751bfde1f4975cdcb78` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.159938 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b4839566df4751bfde1f4975cdcb78)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.162326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.216162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b4839566df4751bfde1f4975cdcb78', 'request_id': 'req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:31.221921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b4839566df4751bfde1f4975cdcb78', 'request_id': 'req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:31.231895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b4839566df4751bfde1f4975cdcb78', 'request_id': 'req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:31.247396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] The token cache is being invalidate because project 13b4839566df4751bfde1f4975cdcb78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:31.247835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:31.248569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5df80724-4d02-4cd3-bb7e-63ae719bc4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:31.250538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1220/2442] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:31 2022] DELETE /identity/v3/projects/13b4839566df4751bfde1f4975cdcb78 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:31.256496 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.256978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.257308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.278121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:31.279366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:31.341715 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1223/2443] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:31.350006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.350297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.350574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.368778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:31.369999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:31.378489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1221/2444] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:31.382789 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.393048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a7374-21bd-4fab-930c-46321ad360b7 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.397571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.398215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.398492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.398762 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.399504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:41:31.399871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.401434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.413453 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1224/2445] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:41:31.417745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.426942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a38b0-cb6e-49fa-823d-a9ca253b0e25 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.431562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.432250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.432537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.433343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.434030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.435964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.451035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:31.463808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1222/2446] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:31 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:31.468003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.478332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dcb62-e878-4c7c-92e7-f0b94e507d43 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.532886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.533468 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.533747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.534019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.534692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.536622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.543611 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:31.557489 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1225/2447] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:31 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:31.562254 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.572935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34d0a2-3590-4fb4-ae6a-fbdeb6bf85d1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.578397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.579052 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.579368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.580047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.580781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.582380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.592927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1223/2448] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:31.596540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.606970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbbddc-312f-4753-909b-ccaed042e2a9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.611687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.612345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.612625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.612899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.613620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.615236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be386229-b294-4d89-8a9a-a589e7405f8f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.625733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1226/2449] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:31.629783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.639731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab59ffa-9ee0-4164-ae20-616365ac6351 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.644294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.645172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.645449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.645751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] PATH_INFO: `/v3/projects/4620033402a34d98814dd490b3ec13dc/users/d3cead37bec04abb82151de765c60a9d/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.659882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4620033402a34d98814dd490b3ec13dc, user_id=d3cead37bec04abb82151de765c60a9d, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.662774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.676130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1224/2450] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:31 2022] PUT /identity/v3/projects/4620033402a34d98814dd490b3ec13dc/users/d3cead37bec04abb82151de765c60a9d/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:31.686713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.686994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.687272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.712794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] MFA Rules not processed for user `d3cead37bec04abb82151de765c60a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:31.713994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:31.776640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1227/2451] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:31.788754 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:31.790411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:31.865703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d576832-7924-4114-b7e1-3e9c5ea6c5d8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:31.872070 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:31.872711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:31.872993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:31.873267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:31.881243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:31.882823 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:31.891640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1225/2452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:33.882287 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:33.884851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:33.909108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fb1a-9ec8-4d50-bc37-574430cac24c None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:33.922935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:33.923747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:33.924136 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:33.924511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:33.927074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:33.941219 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:33.942948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:33.955994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1228/2453] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:34.045712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:34.047166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.133838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248933c-18d1-41d2-861f-f30b680504b8 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:34.139340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:34.140013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:34.140463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:34.140741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:34.142625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.154248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:34.155733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd909e-a6ea-404a-bd47-cc37c23c1d3e service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:34.166073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1226/2454] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:34.611032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:34.683355 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6388ee0-7d30-4e6b-8833-5c615fffcee1 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:34.683895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6388ee0-7d30-4e6b-8833-5c615fffcee1 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:34.684185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6388ee0-7d30-4e6b-8833-5c615fffcee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:34.701539 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d779a14-0c33-4199-bab4-2a0f44fcfb0a service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:34.704418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a6388ee0-7d30-4e6b-8833-5c615fffcee1 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:34.705644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a6388ee0-7d30-4e6b-8833-5c615fffcee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.707893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:34.709019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:34.709498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:34.710114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] PATH_INFO: `/v3/users/d3cead37bec04abb82151de765c60a9d` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:34.711597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3cead37bec04abb82151de765c60a9d)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:34.713996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:34.779651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3cead37bec04abb82151de765c60a9d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:34.791456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:34.791923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3cead37bec04abb82151de765c60a9d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:34.797452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:34.812020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1227/2455] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:41:34.822600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3cead37bec04abb82151de765c60a9d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:34.823578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3cead37bec04abb82151de765c60a9d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:34.841856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3cead37bec04abb82151de765c60a9d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:34.849901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1229/2456] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:34 2022] DELETE /identity/v3/users/d3cead37bec04abb82151de765c60a9d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:34.854258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:34.855069 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:34.855387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:34.869094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:34.872145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.878372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:34.881185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.962638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1228/2457] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:34.964994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75023e52-a87f-45e7-85b8-55875e37f3f3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:34.974801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:34.974978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:34.975807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:34.976165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:34.976978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:34.979265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:34.981333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:35.048901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:35.050610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:35.058032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213d0f-773d-4487-92f2-ed7e70fa698b None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:35.061396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1230/2458] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:41:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:41:35.062189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:35.062923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:35.063404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:35.063691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:35.074163 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:35.075918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a3c01-45f1-400f-8606-12e96a6465b5 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:35.105470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1229/2459] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:35.568218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:35.570527 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:35.597121 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b11f-b371-45ef-846d-7a8cea7af166 service swift] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:35.601922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:35.602917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:35.603269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:35.603651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] PATH_INFO: `/v3/projects/4620033402a34d98814dd490b3ec13dc` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:35.604967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4620033402a34d98814dd490b3ec13dc)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:35.606956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:35.691863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4620033402a34d98814dd490b3ec13dc', 'request_id': 'req-e5ff9d95-33dc-432a-a434-2c4380b94b0d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:35.702398 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4620033402a34d98814dd490b3ec13dc', 'request_id': 'req-e5ff9d95-33dc-432a-a434-2c4380b94b0d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:35.709327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4620033402a34d98814dd490b3ec13dc', 'request_id': 'req-e5ff9d95-33dc-432a-a434-2c4380b94b0d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:35.721118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:35.728503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] The token cache is being invalidate because project 4620033402a34d98814dd490b3ec13dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:35.730998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:35.731367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e5ff9d95-33dc-432a-a434-2c4380b94b0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:35.733493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1231/2460] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:35 2022] DELETE /identity/v3/projects/4620033402a34d98814dd490b3ec13dc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:35.739099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:35.740956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:35.741863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:35.778631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:35.781860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:35.888348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1230/2461] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:35.896917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:35.897221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:35.897737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:35.925802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:35.927739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:35.937580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1232/2462] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:35.941980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:35.953391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e9330-974f-4e9d-9d98-6325c07de64d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:35.968088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:35.968844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:35.969136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:35.969418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:35.970159 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:41:35.970474 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:35.973624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:35.993570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1231/2463] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:41:35.996100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.011786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f0cf3-0c87-4e81-b843-590f82f7723f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:36.017403 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:36.017981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.018431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.019066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.020240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:36.022544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:36.068367 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1233/2464] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:35 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:36.068955 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.109236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed31e0-a2c7-4d1b-b24c-adccc0ed87ce admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:36.172600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:36.291538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:36.292300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.292592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.295529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.299401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:36.301834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:36.354881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1232/2465] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:36 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:41:36.357084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.397799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83103a-1771-4666-af29-380924236be3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:36.435590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:36.436203 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.436487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.436765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.438075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:36.440584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:36.460586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.461059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1234/2466] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:36.475502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051bfdb-b135-4d99-81da-92bd08de08bb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:36.491190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:36.491916 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.492205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.498745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.500339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:36.501879 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31526d54-8a1c-46ce-87fe-a8868b24ca4e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:36.539421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1233/2467] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:41:36.544056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.578104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5e6e1-ed4c-40ec-8571-ec486eef1360 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:36.591935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:36.592872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.593133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.594890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] PATH_INFO: `/v3/projects/0035c298631f46a4a7a3642ab324f863/users/54403dbf6f4b4fa680ff9eddf299d5d8/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.640822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0035c298631f46a4a7a3642ab324f863, user_id=54403dbf6f4b4fa680ff9eddf299d5d8, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:36.653221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:36.693675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:36.693848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1235/2468] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:36 2022] PUT /identity/v3/projects/0035c298631f46a4a7a3642ab324f863/users/54403dbf6f4b4fa680ff9eddf299d5d8/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:36.694571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:36.694861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:36.740583 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] MFA Rules not processed for user `54403dbf6f4b4fa680ff9eddf299d5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:36.741817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:36.951542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:36.966399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1234/2469] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:37.080570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5a7fc-f1e1-4b75-8f8b-649517e690ea admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.086945 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.088551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.088938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.089731 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.090798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.092926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.145816 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1236/2470] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:36 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:37.148713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.164486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845c7f-6b6c-49f7-b160-7e363267b418 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.251719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.252401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.252741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.253028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.253716 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.255705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.282412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1235/2471] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:37 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:37.283822 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.299955 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf0064-30c8-4884-b32d-bb5a48909397 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.305646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.305939 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.306302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.306486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.307183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.309483 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.324857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1237/2472] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:41:37.327489 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.339625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8d0429-5054-45d4-b6a6-d6f132a38a4a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.345216 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.346449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.346904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.347389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] PATH_INFO: `/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319/users/5fd44abc784e4c95a9f2f78246f30a9d/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.365553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbee0ad6ae65427d9f2de452c0ca6319, user_id=5fd44abc784e4c95a9f2f78246f30a9d, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.368630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.388626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:37.399370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1236/2473] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:37 2022] PUT /identity/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319/users/5fd44abc784e4c95a9f2f78246f30a9d/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:37.400992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.414416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f0ca-3721-43da-9b82-da0b963983ae admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.525625 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.526487 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.526847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.527494 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.528442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.530330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34bd36-0559-4d31-911d-a8c6dc29e260 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.543099 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1238/2474] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:41:37.548620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.561387 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1886a-cbd5-4988-ab11-55be16a08625 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.573227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.574515 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.574978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.575261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] PATH_INFO: `/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319/users/5fd44abc784e4c95a9f2f78246f30a9d/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.580180 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbee0ad6ae65427d9f2de452c0ca6319, user_id=5fd44abc784e4c95a9f2f78246f30a9d, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.584496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.610139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1237/2475] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:37 2022] PUT /identity/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319/users/5fd44abc784e4c95a9f2f78246f30a9d/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:37.612549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.613077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.613707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.623630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:37.658452 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] MFA Rules not processed for user `5fd44abc784e4c95a9f2f78246f30a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:37.659721 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:37.787439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1239/2476] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:37.800039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:37.801988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:37.874233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e101-7820-4afd-9732-c410994c8e63 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:37.878428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:37.879103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:37.879435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:37.879719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:37.890295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:37.891788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:37.907921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1238/2477] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:38.022234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:38.024288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:38.041778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5830b-83e5-4b80-9661-c15d43153d23 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:38.046638 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:38.047323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:38.047600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:38.047880 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:38.128037 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:38.129529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846acd-2fcb-45f4-8ac9-f4a0b03ca451 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:38.139961 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1240/2478] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:40.848228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:40.849855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:40.945509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aeba0d-3f8b-4a23-8b6b-e97728165ee8 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:40.952975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:40.953709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:40.954196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:40.954196 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:40.956980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:40.985124 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:40.988934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:41.003350 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1239/2479] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:41.923176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:41.924437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:41.924919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:41.943956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:41.945962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:42.015962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1241/2480] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:41:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:42.032912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:42.035126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:42.122866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3e04-2331-4cf4-aff9-eaa640b0530d service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:42.131189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:42.132307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:42.132729 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:42.133145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:42.143057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:42.144711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779dbe2-ff70-47f8-a86b-260484d5f8aa service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:42.154802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1240/2481] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:41:48.702557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:48.736264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:48.769210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26204229-b5ea-4a63-b09e-c771fa2ca556 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:48.773874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:48.774908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:48.775193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:48.775506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] PATH_INFO: `/v3/users/54403dbf6f4b4fa680ff9eddf299d5d8` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:48.776784 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54403dbf6f4b4fa680ff9eddf299d5d8)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:48.778722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:48.833343 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54403dbf6f4b4fa680ff9eddf299d5d8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:48.842252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54403dbf6f4b4fa680ff9eddf299d5d8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:48.858007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54403dbf6f4b4fa680ff9eddf299d5d8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:48.858650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54403dbf6f4b4fa680ff9eddf299d5d8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:48.867122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54403dbf6f4b4fa680ff9eddf299d5d8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:48.873631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1242/2482] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:48 2022] DELETE /identity/v3/users/54403dbf6f4b4fa680ff9eddf299d5d8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:48.878225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:48.878734 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:48.879221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:48.887428 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:48.902030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:48.903278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:48.971431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1241/2483] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:48.980757 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:48.982676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:49.054151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714faad-2b53-4f7d-8064-fb0f88f552a9 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:49.059179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:49.059892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:49.060170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:49.060446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:49.071966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:49.073701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:49.089577 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1243/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:49.488966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:49.493053 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:49.579248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:49.580756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:49.598636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb887e48-a9aa-4be3-a8ef-63dee1055f58 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:49.605980 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:49.607046 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:49.607474 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:49.607846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] PATH_INFO: `/v3/projects/0035c298631f46a4a7a3642ab324f863` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:49.609288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0035c298631f46a4a7a3642ab324f863)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:49.611458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:49.620483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5079a-fb65-4e26-ab9d-9d7e016728bb None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:49.636959 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:49.639751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:49.640346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:49.641307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:49.647689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:49.665127 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:49.667632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:49.686027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0035c298631f46a4a7a3642ab324f863', 'request_id': 'req-33c49e61-c310-43d4-9a93-2c34386c10a1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.691246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:49.694192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1242/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:49.701009 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0035c298631f46a4a7a3642ab324f863', 'request_id': 'req-33c49e61-c310-43d4-9a93-2c34386c10a1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.710919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0035c298631f46a4a7a3642ab324f863', 'request_id': 'req-33c49e61-c310-43d4-9a93-2c34386c10a1'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.726431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] The token cache is being invalidate because project 0035c298631f46a4a7a3642ab324f863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:49.726963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.728061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.730478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1244/2486] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:49 2022] DELETE /identity/v3/projects/0035c298631f46a4a7a3642ab324f863 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:49.734790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:49.748952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa3f60-a44c-4ffb-8214-7bb99c4712ab service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:49.814399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:49.815592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:49.815876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:49.816150 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] PATH_INFO: `/v3/users/5fd44abc784e4c95a9f2f78246f30a9d` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:49.817869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd44abc784e4c95a9f2f78246f30a9d)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:49.819372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:49.862883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:49.882135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd44abc784e4c95a9f2f78246f30a9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.892785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd44abc784e4c95a9f2f78246f30a9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.893560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd44abc784e4c95a9f2f78246f30a9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.908642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd44abc784e4c95a9f2f78246f30a9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.919574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-008d469f-9871-4056-b7c2-8e0d32773bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd44abc784e4c95a9f2f78246f30a9d'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:49.935824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1243/2487] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:49 2022] DELETE /identity/v3/users/5fd44abc784e4c95a9f2f78246f30a9d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:50.355934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:50.357445 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:50.376923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c49e61-c310-43d4-9a93-2c34386c10a1 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:50.461296 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:50.462637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:50.462999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:50.463385 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] PATH_INFO: `/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:50.465131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbee0ad6ae65427d9f2de452c0ca6319)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:50.467610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:50.542027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbee0ad6ae65427d9f2de452c0ca6319', 'request_id': 'req-10f6ce59-8139-405a-a981-024224351a14'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:50.555736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbee0ad6ae65427d9f2de452c0ca6319', 'request_id': 'req-10f6ce59-8139-405a-a981-024224351a14'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:50.567765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbee0ad6ae65427d9f2de452c0ca6319', 'request_id': 'req-10f6ce59-8139-405a-a981-024224351a14'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:50.587913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] The token cache is being invalidate because project fbee0ad6ae65427d9f2de452c0ca6319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:50.588286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:50.590371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-10f6ce59-8139-405a-a981-024224351a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:50.596467 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1245/2488] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:50 2022] DELETE /identity/v3/projects/fbee0ad6ae65427d9f2de452c0ca6319 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:50.597810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:50.598107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:50.598386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:50.628328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:50.629582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:50.741285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1244/2489] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:41:50.749938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:50.750450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:50.750553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:50.790892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:50.793490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:50.805869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1246/2490] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:50.810203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:50.822913 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925ed94-7170-4fcb-91ed-fec8a927a6a7 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:50.836368 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:50.840962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:50.841305 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:50.841601 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:50.842449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:41:50.842741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:50.844870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:50.861720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1245/2491] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:41:50.866101 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:50.876791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f38b8b-9e83-403e-9508-6e76550b9d19 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:50.883055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:50.883981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:50.884426 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:50.884859 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:50.885862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:50.888082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:50.929872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:50.930881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1247/2492] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:50 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:50.940950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5d6ff-652d-4a72-9f62-99315410b668 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.011866 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.013104 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.014282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.015272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.016881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.019036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.061492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1246/2493] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:50 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:41:51.065634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.086388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c6d0-469f-4e31-9109-226440d09e76 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.098664 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.099510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.100599 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.101019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.101696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.116987 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.133448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1248/2494] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:51.138001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.154760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4173f1-5a5d-472a-be73-21f90f2ad01e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.163349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.164258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.164547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.164827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.165603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.168154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb9b33-e553-4f33-890a-d89a55261069 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.185912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1247/2495] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:51.192396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.206703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed026f-a579-439c-8455-22760aa24613 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.212390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.214261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.214261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.214608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] PATH_INFO: `/v3/projects/015669ab8e2448a987aba1ffc94e15c4/users/43d18e07a7804469ae8e721cf959f93a/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.250856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=015669ab8e2448a987aba1ffc94e15c4, user_id=43d18e07a7804469ae8e721cf959f93a, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.253709 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.270990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1249/2496] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:51 2022] PUT /identity/v3/projects/015669ab8e2448a987aba1ffc94e15c4/users/43d18e07a7804469ae8e721cf959f93a/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:51.282310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.282612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.282883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.318349 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:51.326638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] MFA Rules not processed for user `43d18e07a7804469ae8e721cf959f93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:51.328182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:51.430597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1248/2497] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:51.449077 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.452840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:51.465455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:51.561351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.562787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:51.582817 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b251b1-f270-4b2a-b369-dd5b0874ee9d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.587262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.587975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.588284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.589063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.607518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.613303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.634330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1250/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:51.658867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:51.660701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:51.674261 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326c55-8e93-417d-aa91-0654e071a4ad None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.681512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.683624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.683624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.683851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] PATH_INFO: `/v3/users/e4f9820744874f9c979cb18cbf624fb4` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.685385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f9820744874f9c979cb18cbf624fb4)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.687436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.757184 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f9820744874f9c979cb18cbf624fb4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:51.771552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe66000-7002-44d5-915e-89d23950cf79 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:51.773883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f9820744874f9c979cb18cbf624fb4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:51.774709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f9820744874f9c979cb18cbf624fb4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:51.777263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:51.778008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:51.778300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:51.778580 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:51.780466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:51.792377 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:51.794431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:51.803386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f9820744874f9c979cb18cbf624fb4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:51.807735 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1251/2499] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:51.816954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f9820744874f9c979cb18cbf624fb4'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:51.826088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1249/2500] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:51 2022] DELETE /identity/v3/users/e4f9820744874f9c979cb18cbf624fb4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:41:52.254188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:52.256006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:52.272160 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6632a07-fca3-4750-98ef-72dc5c04c92a service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:52.352684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:52.353555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:52.353839 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:52.354885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] PATH_INFO: `/v3/projects/65ccb95f08764f99b5a5c1166c42410b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:52.356243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ccb95f08764f99b5a5c1166c42410b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:52.358169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:52.359207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:52.362519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:52.391028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:52.408767 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ccb95f08764f99b5a5c1166c42410b', 'request_id': 'req-a75c270c-9f7d-4f27-9da0-85e423d7745b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.412115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:52.418554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ccb95f08764f99b5a5c1166c42410b', 'request_id': 'req-a75c270c-9f7d-4f27-9da0-85e423d7745b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.425115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ccb95f08764f99b5a5c1166c42410b', 'request_id': 'req-a75c270c-9f7d-4f27-9da0-85e423d7745b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.440323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c615d5-50ef-4fb3-bbe1-30914ebe5ca0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:52.443461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] The token cache is being invalidate because project 65ccb95f08764f99b5a5c1166c42410b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:52.443760 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.444415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.446303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1252/2501] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:52 2022] DELETE /identity/v3/projects/65ccb95f08764f99b5a5c1166c42410b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:52.450683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:52.452089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:52.504680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:52.505384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:52.505671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:52.505949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:52.507897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:52.520449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:52.522038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:52.525263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c270c-9f7d-4f27-9da0-85e423d7745b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:52.531529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:52.532411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:52.532691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:52.532963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] PATH_INFO: `/v3/users/a2e1c0d758534efa8c04869bc54ee2aa` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:52.533978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1250/2502] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:41:52.534184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e1c0d758534efa8c04869bc54ee2aa)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:52.536133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:52.585914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e1c0d758534efa8c04869bc54ee2aa'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.593472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e1c0d758534efa8c04869bc54ee2aa'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.606796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e1c0d758534efa8c04869bc54ee2aa'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.607488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e1c0d758534efa8c04869bc54ee2aa'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.614795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-23f5cc23-ef34-4dd3-988b-387da34f9aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e1c0d758534efa8c04869bc54ee2aa'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.622359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1253/2503] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:41:52 2022] DELETE /identity/v3/users/a2e1c0d758534efa8c04869bc54ee2aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:41:52.825780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:52.840446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81094832-d1a3-4c87-bf12-d3b5df13dd6d service placement] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:52.913925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:52.914801 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:52.915085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:52.915457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] PATH_INFO: `/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:52.917236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e77b1ed93048a8a6bfda9c5976ae96)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:52.919139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:52.968251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e77b1ed93048a8a6bfda9c5976ae96', 'request_id': 'req-f5528ba8-0296-43b0-9160-c6c4bcf8f924'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.973043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:52.974617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e77b1ed93048a8a6bfda9c5976ae96', 'request_id': 'req-f5528ba8-0296-43b0-9160-c6c4bcf8f924'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.983404 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e77b1ed93048a8a6bfda9c5976ae96', 'request_id': 'req-f5528ba8-0296-43b0-9160-c6c4bcf8f924'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.995602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] The token cache is being invalidate because project 41e77b1ed93048a8a6bfda9c5976ae96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:41:52.995890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.996541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f5528ba8-0296-43b0-9160-c6c4bcf8f924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:41:52.998089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1251/2504] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:41:52 2022] DELETE /identity/v3/projects/41e77b1ed93048a8a6bfda9c5976ae96 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:53.003165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.003703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.003924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.023970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:53.025113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:53.080921 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:53.091245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1254/2505] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:41:53.098741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.099038 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.099346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.117997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:53.119229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:53.127397 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1252/2506] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:41:53.131836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.141561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d219a-7569-40ba-b19d-400c0d0412f6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.146390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.147058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.147373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.147655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.148379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:41:53.148688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.150302 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.162983 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1255/2507] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:41:53.167984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.177267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127ef52-5d33-4a98-abf5-d05b8ccc1e19 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.182068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.182726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.183016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.183257 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.183980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.185872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.211751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1253/2508] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:53.216640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.227591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6074b-c63a-47eb-996c-ed0dfeb36411 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.309514 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.310540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.310953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.311378 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.312152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.314176 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.344054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1256/2509] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:53 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:41:53.348114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.360550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ec0d0-fecc-460d-8003-db20bd9bc136 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.368299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.369059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.369403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.370445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.370568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.372446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.396706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1254/2510] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:53.398096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.421394 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563bbc77-0129-4771-bf82-e614b444e477 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.429212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.429923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.430204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.430502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.431393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.434109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0dc69-1e40-4e20-ba8c-d8386b709db7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.456272 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1257/2511] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:53.460246 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.471361 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60de3f0-60a5-46d9-b9c5-b18194776f71 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.477454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.478473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.478828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.479770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] PATH_INFO: `/v3/projects/ee1753001c624ae3941cfc51b3d9d6ef/users/0821d7c651dc4571834b8f3c40b225a9/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.496442 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1753001c624ae3941cfc51b3d9d6ef, user_id=0821d7c651dc4571834b8f3c40b225a9, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.499206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.522113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1255/2512] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:53 2022] PUT /identity/v3/projects/ee1753001c624ae3941cfc51b3d9d6ef/users/0821d7c651dc4571834b8f3c40b225a9/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:41:53.527749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a5fad-f21e-40e1-ac8d-5d2887e0e433 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.528043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a5fad-f21e-40e1-ac8d-5d2887e0e433 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.528322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a5fad-f21e-40e1-ac8d-5d2887e0e433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.551606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-5a1a5fad-f21e-40e1-ac8d-5d2887e0e433 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:53.553143 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5a1a5fad-f21e-40e1-ac8d-5d2887e0e433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:53.626409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1258/2513] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:53.637454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:53.639268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:53.719419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca00285-bb9f-4339-b47e-bfd4bf0bce12 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:53.723269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:53.723965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:53.724876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:53.725167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:53.735741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:53.737153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:53.748640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1256/2514] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:41:56.938328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607826b-807f-4f2e-b40f-48f0f7485781 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:56.938862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607826b-807f-4f2e-b40f-48f0f7485781 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:56.938862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607826b-807f-4f2e-b40f-48f0f7485781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:56.972666 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1607826b-807f-4f2e-b40f-48f0f7485781 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:56.973930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1607826b-807f-4f2e-b40f-48f0f7485781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:57.083570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1259/2515] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:41:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:41:57.100458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:57.102105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:57.217367 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a80d7-678f-4025-8d38-3a944caf1d2d service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:57.223408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:57.240929 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:57.241262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:57.241553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:57.252550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:57.254234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:57.267087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1257/2516] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:41:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:41:57.827800 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:57.828165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:57.828510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:57.852706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:57.862841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] MFA Rules not processed for user `0821d7c651dc4571834b8f3c40b225a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:41:57.864533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:41:57.936513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1260/2517] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:41:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:41:57.938147 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:57.951897 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ee073-e5f3-4fe0-b364-7ae0e5035f5d service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:57.959229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:57.960403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:57.960816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:57.961240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:57.962151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:57.964591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:57.978680 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:41:58.012660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1258/2518] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:41:57 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:41:58.013491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:58.038376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef449d-09e1-4b9f-9af2-b111732906bc None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:58.155667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:58.156541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:58.156832 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:58.157106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:58.160078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:58.164774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:58.218836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:58.224528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1261/2519] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:41:58 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:41:58.231325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411097a-f555-48e7-a9f5-ecd29e14e510 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:58.236112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:58.236841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:58.237332 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:58.237606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:58.238611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:58.240505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:58.258093 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:58.263890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1259/2520] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:58.270041 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd9ea8-c7a6-46fe-b671-b4ddef5139b2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:58.275604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:58.277043 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:58.277401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:58.277678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] PATH_INFO: `/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c/users/b6ad8843675246bcba9b714502e63bf8/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:58.304694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ef7546dc454ebaa3a3ef9d2f11e36c, user_id=b6ad8843675246bcba9b714502e63bf8, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:58.311819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:58.332551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1262/2521] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:58 2022] PUT /identity/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c/users/b6ad8843675246bcba9b714502e63bf8/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:41:58.336765 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:58.351206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef6fe-6f3a-42de-940c-0bbe2d2c8375 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:58.497922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:58.499338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:58.499338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:58.499711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:58.500530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:58.502245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c049932-7d59-4f3b-a3ca-8aa829c7de85 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:58.514945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1260/2522] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:41:58.519250 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:41:58.538962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed3706-e574-4d24-9cdb-f4a7c5e28b67 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:41:58.548019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:41:58.548879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:41:58.549172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:41:58.549428 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] PATH_INFO: `/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c/users/b6ad8843675246bcba9b714502e63bf8/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:41:58.552845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ef7546dc454ebaa3a3ef9d2f11e36c, user_id=b6ad8843675246bcba9b714502e63bf8, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:41:58.555582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:41:58.613565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1263/2523] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:41:58 2022] PUT /identity/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c/users/b6ad8843675246bcba9b714502e63bf8/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:06.139702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b941b7-093e-4aee-96ce-4ad8527c8623 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:06.140242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b941b7-093e-4aee-96ce-4ad8527c8623 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:06.140242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b941b7-093e-4aee-96ce-4ad8527c8623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:06.174058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c3b941b7-093e-4aee-96ce-4ad8527c8623 None None] MFA Rules not processed for user `b6ad8843675246bcba9b714502e63bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:06.175310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c3b941b7-093e-4aee-96ce-4ad8527c8623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:06.254881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1261/2524] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:06.270887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:06.272749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:06.342100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:06.373315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ac0e2-9c4c-4a92-851d-3c42fa250efb admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:06.379493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:06.380229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:06.380522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:06.380799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:06.445914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:06.447830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:06.458392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1264/2525] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:08.009946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190419-453e-410d-9ff9-a0d3a2bdbf1b None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:08.010472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190419-453e-410d-9ff9-a0d3a2bdbf1b None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:08.011333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190419-453e-410d-9ff9-a0d3a2bdbf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:08.034892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-35190419-453e-410d-9ff9-a0d3a2bdbf1b None None] MFA Rules not processed for user `0821d7c651dc4571834b8f3c40b225a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:08.036444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-35190419-453e-410d-9ff9-a0d3a2bdbf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:08.049419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1262/2526] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:13.043764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:13.045216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.123793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8fbe6-86b7-4755-9387-e8c875b45cef service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:13.128970 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:13.130003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.130386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.130756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] PATH_INFO: `/v3/users/43d18e07a7804469ae8e721cf959f93a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.132094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d18e07a7804469ae8e721cf959f93a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:13.134110 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:13.191878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d18e07a7804469ae8e721cf959f93a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.202035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d18e07a7804469ae8e721cf959f93a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.218073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d18e07a7804469ae8e721cf959f93a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.218882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d18e07a7804469ae8e721cf959f93a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.232903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d18e07a7804469ae8e721cf959f93a'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.242606 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1265/2527] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:13 2022] DELETE /identity/v3/users/43d18e07a7804469ae8e721cf959f93a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:13.247719 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.248230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.248230 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.267602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:13.268921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.318921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:13.332030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1263/2528] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:13.341791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:13.343330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.355165 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:13.413212 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc736da-19d0-417a-8985-39eb746db24d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:13.417546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:13.418269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.418621 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.418956 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.428470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:13.430044 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e200e-aba0-49ac-98fb-31b6e341d51b service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:13.440615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1266/2529] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:13.756343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:13.757908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.771384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16010c59-8322-45a4-bd38-4db4e2b584fc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:13.776876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:13.777836 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.778325 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.778809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] PATH_INFO: `/v3/projects/015669ab8e2448a987aba1ffc94e15c4` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.780450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=015669ab8e2448a987aba1ffc94e15c4)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:13.782598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:13.833907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015669ab8e2448a987aba1ffc94e15c4', 'request_id': 'req-65dd361e-60bf-458f-8d86-8dd8e914a55a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.840502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015669ab8e2448a987aba1ffc94e15c4', 'request_id': 'req-65dd361e-60bf-458f-8d86-8dd8e914a55a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.848155 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015669ab8e2448a987aba1ffc94e15c4', 'request_id': 'req-65dd361e-60bf-458f-8d86-8dd8e914a55a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.860408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] The token cache is being invalidate because project 015669ab8e2448a987aba1ffc94e15c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:13.860695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.861348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-65dd361e-60bf-458f-8d86-8dd8e914a55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:13.863114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1264/2530] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:13 2022] DELETE /identity/v3/projects/015669ab8e2448a987aba1ffc94e15c4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:13.870950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321d80-9775-4a33-bedc-489699137214 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.871304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321d80-9775-4a33-bedc-489699137214 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.871670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321d80-9775-4a33-bedc-489699137214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.891971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-e4321d80-9775-4a33-bedc-489699137214 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:13.893164 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-e4321d80-9775-4a33-bedc-489699137214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.954678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1267/2531] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:13.962676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:13.962964 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:13.963240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:13.981259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:13.982624 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:13.993088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1265/2532] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:13.999947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4321d80-9775-4a33-bedc-489699137214 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.012243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4321d80-9775-4a33-bedc-489699137214 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.016962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.017628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.017912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.018184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.018912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:14.019244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.020881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.033034 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1268/2533] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:42:14.037625 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.046736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57634929-4bb7-4897-ae40-c975da0ad22c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.051258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.051923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.052203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.052470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.053137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.055008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.080306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1266/2534] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/projects => generated 375 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:14.084318 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.094546 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26320697-0100-47ac-98ca-1bd44dc69fc5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.150155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.150959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.151248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.151561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.152242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.154157 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.174475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1269/2535] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:14.181048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.194315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572a4c-0772-4e14-af2c-53d35360d8a8 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.199575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.200239 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.200519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.200795 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.201514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.203168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.213818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1267/2536] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:14.217783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.227985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b105-0b06-4348-8523-269d821f8c96 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.232978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.233641 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.233923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.234197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.234948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.236583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde1dee-a83f-4583-963c-c4c473428d40 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.248248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1270/2537] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:14.255704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.270229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5a03e-b825-42b6-a783-ed66cc302b4e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.276824 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.278187 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.278483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.278777 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] PATH_INFO: `/v3/projects/df4fde55745a449e8ec1e92d41adac3b/users/70e1638f68d6450aa4e9e4f255f2d6d8/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.295747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4fde55745a449e8ec1e92d41adac3b, user_id=70e1638f68d6450aa4e9e4f255f2d6d8, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.298440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.324139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1268/2538] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:14 2022] PUT /identity/v3/projects/df4fde55745a449e8ec1e92d41adac3b/users/70e1638f68d6450aa4e9e4f255f2d6d8/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:14.335603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.335903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.336179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.367274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] MFA Rules not processed for user `70e1638f68d6450aa4e9e4f255f2d6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:14.368559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:14.378066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:14.466204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1271/2539] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:14.479709 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.562416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68e7e4-ed8e-4e0c-8e3f-8f0cb95f3b72 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.568418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.569973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.570256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.571328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.572030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.574638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.600448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:14.612563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1269/2540] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:14.615412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.635658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331cea3-2a80-4bf5-9147-c294ac4aeada None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.738510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.739196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.739649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.739930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.740722 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.742686 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.768978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1272/2541] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:14.770844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.784242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb0ef8-5993-4fe4-966e-de4e14d036e7 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.793554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.793993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.794223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.794495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.795324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.797232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.811262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1270/2542] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:14.816231 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.828626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2525e67-b297-4cb5-b74c-a6a7c58746b7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.833679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.834374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.834657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.835054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.835699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.837475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec286f1-ad3b-4fe8-87d2-684a5eaf81e0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.852778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1273/2543] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:14.855315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:14.865832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f044bf-85fe-465f-95a0-649fed156a75 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:14.870998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:14.871977 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.872265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.872545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] PATH_INFO: `/v3/projects/473de344d49a4e4a9c847ab33e39abbf/users/9c49e6e9883b4ab396dfc750b71a2137/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.897713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473de344d49a4e4a9c847ab33e39abbf, user_id=9c49e6e9883b4ab396dfc750b71a2137, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:14.901032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:14.923815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1271/2544] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:14 2022] PUT /identity/v3/projects/473de344d49a4e4a9c847ab33e39abbf/users/9c49e6e9883b4ab396dfc750b71a2137/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:14.930161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:14.930568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:14.930764 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:14.969400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] MFA Rules not processed for user `9c49e6e9883b4ab396dfc750b71a2137`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:14.970633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:15.050064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1274/2545] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:15.055870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.133453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19a4bd-8d3f-4699-9bf3-b02df291fc40 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.143512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.144237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.144525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.144815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.145552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.147505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.175382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1272/2546] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:15 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:15.180075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.192513 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd9b2b-ea57-410b-ab81-9b424a544167 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.209701 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:15.263448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.264128 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.264408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.264682 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.265374 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.267410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.288178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1275/2547] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:15 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:15.292329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.302458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efa49-5270-403f-838b-523b9142bf99 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.307630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.308264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.308549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.308835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.309552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.311145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.322616 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1273/2548] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:15.326656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.336874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5b28e-bd28-4a25-a966-cbfb4504e7d8 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.342571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.344230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.344400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.344595 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] PATH_INFO: `/v3/projects/1e12047a2a9e4edb932741caba051d76/users/b2f95ec67e934091ae8aa29e2193684c/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.363448 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e12047a2a9e4edb932741caba051d76, user_id=b2f95ec67e934091ae8aa29e2193684c, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.366129 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.380115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1276/2549] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:15 2022] PUT /identity/v3/projects/1e12047a2a9e4edb932741caba051d76/users/b2f95ec67e934091ae8aa29e2193684c/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:15.384302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.394338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7192f8-c66c-42db-9936-dcad1a8b251d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.450834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.451540 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.451899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.452102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.452844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.454440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0bc0b-e830-4272-a8b8-e1dbae13131f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.465311 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1274/2550] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:15.469692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.482670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0768f-8345-468b-8796-39a82a540d77 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.488197 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.489144 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.489510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.489662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] PATH_INFO: `/v3/projects/1e12047a2a9e4edb932741caba051d76/users/b2f95ec67e934091ae8aa29e2193684c/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.492275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e12047a2a9e4edb932741caba051d76, user_id=b2f95ec67e934091ae8aa29e2193684c, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.495211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.506875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1277/2551] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:15 2022] PUT /identity/v3/projects/1e12047a2a9e4edb932741caba051d76/users/b2f95ec67e934091ae8aa29e2193684c/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:15.517299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.517877 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.518262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.549667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] MFA Rules not processed for user `b2f95ec67e934091ae8aa29e2193684c`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:15.550925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:15.584158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:15.635407 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1275/2552] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:15.647608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:15.649350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:15.715058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65d28c-4062-4661-924d-0a08b321e417 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:15.718783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:15.719575 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:15.719868 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:15.720159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:15.777693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:15.779140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b70abc-65fc-4f3f-95db-8955195e64fa service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:15.788532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1278/2553] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:17.338943 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:17.348662 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb711a-3011-47b0-8745-4020310bef12 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:17.352191 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:17.352828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:17.353115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:17.353995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:17.414988 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:17.416930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:17.426965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1276/2554] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:17.799216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:17.799743 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:17.800084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:17.823836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:17.834502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:17.864615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:17.926463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1279/2555] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:42:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:17.940948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:17.943065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:18.074400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cd01-66b5-4c7a-89cc-87c118d929a5 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:18.096280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:18.096912 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:18.097335 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:18.097614 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:18.113517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:18.115349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:18.125711 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:18.127535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:18.134412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a08d02-3ab9-4de7-a834-ad89c4310a2d None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:18.138139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1277/2556] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:42:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:42:18.143388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:18.144516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:18.144948 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:18.145370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:18.166240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:18.167736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8799138-8f0c-4919-9df9-6f26e61fcba1 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:18.178706 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1280/2557] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:42:22.049514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:22.106629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:22.121405 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ecc9f-f460-4021-9577-0e6cce2480ab service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:22.126130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:22.127002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.127303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.127580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] PATH_INFO: `/v3/users/70e1638f68d6450aa4e9e4f255f2d6d8` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.128751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70e1638f68d6450aa4e9e4f255f2d6d8)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:22.130642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:22.175535 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70e1638f68d6450aa4e9e4f255f2d6d8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.183418 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70e1638f68d6450aa4e9e4f255f2d6d8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.184049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70e1638f68d6450aa4e9e4f255f2d6d8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.195475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70e1638f68d6450aa4e9e4f255f2d6d8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.202746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70e1638f68d6450aa4e9e4f255f2d6d8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.209333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1278/2558] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:22 2022] DELETE /identity/v3/users/70e1638f68d6450aa4e9e4f255f2d6d8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:22.214399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.214873 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.214973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.238624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:22.239774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:22.298189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1281/2559] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:22.308750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:22.310420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:22.369531 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b251b-0d93-48e0-95b9-d90f3778d267 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:22.373295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:22.373970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.374250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.374526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.383395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:22.385608 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:22.394883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1279/2560] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:42:22.663521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:22.665339 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:22.680138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032120eb-0cb2-4bce-a92f-1a6e357b9040 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:22.686226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:22.687341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.687708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.688097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] PATH_INFO: `/v3/projects/df4fde55745a449e8ec1e92d41adac3b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.695397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4fde55745a449e8ec1e92d41adac3b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:22.698863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:22.738956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:22.748547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597dca1-b4f4-4bdf-8366-19987e855d78 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:22.754888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4fde55745a449e8ec1e92d41adac3b', 'request_id': 'req-206162cf-4698-4d9e-a939-7a0d02880b80'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.762871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4fde55745a449e8ec1e92d41adac3b', 'request_id': 'req-206162cf-4698-4d9e-a939-7a0d02880b80'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.769281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4fde55745a449e8ec1e92d41adac3b', 'request_id': 'req-206162cf-4698-4d9e-a939-7a0d02880b80'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.780989 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] The token cache is being invalidate because project df4fde55745a449e8ec1e92d41adac3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:22.781267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.781960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.783558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1282/2561] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:22 2022] DELETE /identity/v3/projects/df4fde55745a449e8ec1e92d41adac3b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:42:22.787258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:22.788836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:22.801259 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:22.802109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.802401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.802678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.804534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:22.830521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:22.830726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:22.857997 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206162cf-4698-4d9e-a939-7a0d02880b80 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:22.862788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:22.863690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:22.863971 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:22.864245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] PATH_INFO: `/v3/users/9c49e6e9883b4ab396dfc750b71a2137` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:22.865447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c49e6e9883b4ab396dfc750b71a2137)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:22.867414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:22.872273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:22.873699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:22.883573 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1280/2562] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:42:22.914828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c49e6e9883b4ab396dfc750b71a2137'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.922907 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c49e6e9883b4ab396dfc750b71a2137'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.935386 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c49e6e9883b4ab396dfc750b71a2137'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.936094 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c49e6e9883b4ab396dfc750b71a2137'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.942998 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c49e6e9883b4ab396dfc750b71a2137'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:22.949107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1283/2563] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:22 2022] DELETE /identity/v3/users/9c49e6e9883b4ab396dfc750b71a2137 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:42:23.397673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:23.410840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e53c0-0be4-4109-a9b0-6f61c9cb807f service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:23.497590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:23.498460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:23.498742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:23.499015 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] PATH_INFO: `/v3/projects/473de344d49a4e4a9c847ab33e39abbf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:23.500251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=473de344d49a4e4a9c847ab33e39abbf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:23.502162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:23.549247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473de344d49a4e4a9c847ab33e39abbf', 'request_id': 'req-5a4827f4-ab50-4494-a556-7ebfdc19b149'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.555356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473de344d49a4e4a9c847ab33e39abbf', 'request_id': 'req-5a4827f4-ab50-4494-a556-7ebfdc19b149'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.563545 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473de344d49a4e4a9c847ab33e39abbf', 'request_id': 'req-5a4827f4-ab50-4494-a556-7ebfdc19b149'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.576454 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] The token cache is being invalidate because project 473de344d49a4e4a9c847ab33e39abbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:23.576747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.577400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.578967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1281/2564] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:23 2022] DELETE /identity/v3/projects/473de344d49a4e4a9c847ab33e39abbf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:23.583028 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:23.584470 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:23.597778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890ba5b-2fc0-413f-b40d-c1016fadcb3f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:23.656230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:23.657336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:23.657612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:23.657893 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] PATH_INFO: `/v3/users/b2f95ec67e934091ae8aa29e2193684c` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:23.659103 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f95ec67e934091ae8aa29e2193684c)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:23.661079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:23.711021 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f95ec67e934091ae8aa29e2193684c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.719630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f95ec67e934091ae8aa29e2193684c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.725459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:23.736678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f95ec67e934091ae8aa29e2193684c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.737541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f95ec67e934091ae8aa29e2193684c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.745592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-452d57b5-d701-4ae0-aa55-02758f70e5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f95ec67e934091ae8aa29e2193684c'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:23.751840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1284/2565] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:23 2022] DELETE /identity/v3/users/b2f95ec67e934091ae8aa29e2193684c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:24.331343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.332769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:24.345303 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4827f4-ab50-4494-a556-7ebfdc19b149 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.404695 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:24.416878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.417692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.417973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.418247 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] PATH_INFO: `/v3/projects/1e12047a2a9e4edb932741caba051d76` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.419490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e12047a2a9e4edb932741caba051d76)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.421575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.488785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e12047a2a9e4edb932741caba051d76', 'request_id': 'req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:24.494172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e12047a2a9e4edb932741caba051d76', 'request_id': 'req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:24.503013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e12047a2a9e4edb932741caba051d76', 'request_id': 'req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:24.514256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] The token cache is being invalidate because project 1e12047a2a9e4edb932741caba051d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:24.514541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:24.515206 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1d393616-0e49-49fd-b7d3-c2c6d6f3b945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:24.516789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1282/2566] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:24 2022] DELETE /identity/v3/projects/1e12047a2a9e4edb932741caba051d76 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:24.521637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.521924 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.522201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.542765 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:24.544000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:24.623507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1285/2567] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:24.631756 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.632051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.632333 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.652890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:24.654118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:24.662730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1283/2568] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:24.667654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.679322 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30d4ff-f12d-461b-8246-1eff3a103b7e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.684787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.685304 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.685588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.685861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.686585 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:24.686894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.688541 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.704365 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1286/2569] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:24.706164 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.717273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e881f8-14d2-4584-a5f6-6be6d7888a65 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.725553 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.726498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.726937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.727413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.729032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.731379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.765685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1284/2570] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:24 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:24.769881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.781877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabdcf0-2992-434c-aa65-82602ffc10e3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.854554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.855317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.855604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.855878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.856568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.858525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.880894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1287/2571] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:24 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:24.885139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.897595 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae05064-0d3b-4169-b7a7-e24cc73880a1 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.903568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.904396 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.904781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.905156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.906024 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.907783 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.920203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1285/2572] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:24.924050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.936679 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b325e4b-bda3-4b2c-b4cc-432e4cf6e856 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.941498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.942251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.942602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.943113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.943958 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:24.945624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9b84-5e21-4659-82bd-9d6948ad6910 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:24.957654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1288/2573] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:42:24.961691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:24.973056 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031cb91-7199-4731-8243-06696e01e822 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:24.979128 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:24.980087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:24.980370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:24.980645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] PATH_INFO: `/v3/projects/4f8dfa60d69d45c29013dbbe6c1dd226/users/354e3e9955a74c1a86b7e4e6c9bbd512/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:24.997434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8dfa60d69d45c29013dbbe6c1dd226, user_id=354e3e9955a74c1a86b7e4e6c9bbd512, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:25.000190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:25.017141 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1286/2574] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:24 2022] PUT /identity/v3/projects/4f8dfa60d69d45c29013dbbe6c1dd226/users/354e3e9955a74c1a86b7e4e6c9bbd512/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:25.023227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:25.023861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:25.024696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:25.047684 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:25.048905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:25.098402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:25.126156 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1289/2575] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:25.136885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:25.138693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:25.211852 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f223f-067f-4747-9d45-53d6a31389b4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:25.217538 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:25.218330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:25.218629 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:25.218976 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:25.230079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:25.231660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343aacc-5d91-478b-8e36-b230e241ca4a service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:25.242882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1287/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:26.118350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:26.131618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299a542-b5d2-4906-a4ef-94e49e47d56e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:26.137352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:26.138060 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:26.138342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:26.138792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:26.140926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:26.154370 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:26.158019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:26.170906 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1290/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:26.214793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:26.215346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:26.215346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:26.265115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] MFA Rules not processed for user `354e3e9955a74c1a86b7e4e6c9bbd512`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:26.266715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:26.289145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:26.391792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1288/2578] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:26.403560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:26.405072 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:26.510668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015053ab-d22f-4d80-9a54-028f3dcf757b service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:26.514739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:26.516254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:26.516725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:26.517436 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:26.528357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:26.529865 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:26.540240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1291/2579] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:26.688421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:26.689779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:26.708294 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50bdf3-321a-45f9-bed4-9db328fc0bfd None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:26.713699 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:26.714768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:26.715047 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:26.715377 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] PATH_INFO: `/v3/users/ee885a29e6fa4b1980d7ae1d03b2b476` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:26.717370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee885a29e6fa4b1980d7ae1d03b2b476)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:26.720317 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:26.792892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee885a29e6fa4b1980d7ae1d03b2b476'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:26.803089 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee885a29e6fa4b1980d7ae1d03b2b476'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:26.805450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee885a29e6fa4b1980d7ae1d03b2b476'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:26.848907 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee885a29e6fa4b1980d7ae1d03b2b476'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:26.872828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ccd9d227-94d3-42f6-9d15-510bd541f095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee885a29e6fa4b1980d7ae1d03b2b476'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:26.881297 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1289/2580] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:26 2022] DELETE /identity/v3/users/ee885a29e6fa4b1980d7ae1d03b2b476 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:27.116127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.128353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9e38e-ef5c-4545-8ba2-9c7712ee14d8 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.220333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.221127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.221411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.221687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] PATH_INFO: `/v3/projects/e068f9b959004be4ad9c778d9010561a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.222918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e068f9b959004be4ad9c778d9010561a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.224892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.292623 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e068f9b959004be4ad9c778d9010561a', 'request_id': 'req-d65d1825-9624-4ffe-8d4d-1336d7040e27'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:27.303791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e068f9b959004be4ad9c778d9010561a', 'request_id': 'req-d65d1825-9624-4ffe-8d4d-1336d7040e27'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:27.312830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e068f9b959004be4ad9c778d9010561a', 'request_id': 'req-d65d1825-9624-4ffe-8d4d-1336d7040e27'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:27.325188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] The token cache is being invalidate because project e068f9b959004be4ad9c778d9010561a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:27.325471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:27.326133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-d65d1825-9624-4ffe-8d4d-1336d7040e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:27.327687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1292/2581] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:27 2022] DELETE /identity/v3/projects/e068f9b959004be4ad9c778d9010561a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:27.332702 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.333175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.333280 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.351495 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:27.354237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:27.425004 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1290/2582] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:27.433433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.433771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.434091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.440751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:27.455032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:27.457055 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:27.465624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1293/2583] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:27.470563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.480581 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107865-9a7f-45b4-992e-d1d71ffa140f None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.485132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.485785 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.486071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.486344 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.487058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:27.487390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.488967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.501209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1291/2584] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:27.505499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.515553 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875b4db-41d3-492b-a842-4e476bf638ec None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.520271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.520922 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.521204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.521485 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.522177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.524181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.549482 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1294/2585] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:27 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:27.553345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.564003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c5564-dd04-412d-a1ea-430879fbfb8f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.568676 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:27.624536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.625202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.625483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.625755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.626460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.628379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.648171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1292/2586] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:27 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:27.652704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.663545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac06a2f-e44a-4744-9170-f058bd657cef admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.668401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.669054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.669335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.669609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.670323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.671943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.683006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1295/2587] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:27.686894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.698458 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3495f33-cf52-4ada-bf78-4ac51b27f80d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.703568 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.704236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.704546 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.704827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.705554 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.707170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228d114-d032-4517-8bbd-0e8311768aca admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.721981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1293/2588] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:27.723057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.734885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d59c8-c10e-4908-baa4-cdd1ae95dec3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:27.739890 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:27.740771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.741051 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.741323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] PATH_INFO: `/v3/projects/c32d8605089a46a4a902e6b3a3590bb8/users/64ac40357265482ab0c20f4c840b7721/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.758079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32d8605089a46a4a902e6b3a3590bb8, user_id=64ac40357265482ab0c20f4c840b7721, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:27.761522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:27.775410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1296/2589] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:27 2022] PUT /identity/v3/projects/c32d8605089a46a4a902e6b3a3590bb8/users/64ac40357265482ab0c20f4c840b7721/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:42:27.786764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8922-c014-4d40-8468-eacb3ff5f67f None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:27.787059 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8922-c014-4d40-8468-eacb3ff5f67f None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:27.787362 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff8922-c014-4d40-8468-eacb3ff5f67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:27.824471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-beff8922-c014-4d40-8468-eacb3ff5f67f None None] MFA Rules not processed for user `64ac40357265482ab0c20f4c840b7721`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:27.825770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-beff8922-c014-4d40-8468-eacb3ff5f67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:27.911994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1294/2590] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:27.924676 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:27.926465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:28.052326 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514c768-45ff-4000-81b2-bb3dc2d015ed admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:28.059147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:28.060224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:28.060818 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:28.061383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:28.073133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:28.075079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:28.079525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9601078-f35a-413d-89eb-c39fe55dbc0c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:28.079869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9601078-f35a-413d-89eb-c39fe55dbc0c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:28.080156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9601078-f35a-413d-89eb-c39fe55dbc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:28.086320 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1297/2591] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:28.100412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a9601078-f35a-413d-89eb-c39fe55dbc0c None None] MFA Rules not processed for user `354e3e9955a74c1a86b7e4e6c9bbd512`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:28.101623 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a9601078-f35a-413d-89eb-c39fe55dbc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:28.170707 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1295/2592] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:35.509392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:35.568804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a55796-ff4a-4947-a6d5-98662e608bf6 service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:35.573406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:35.574254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:35.574532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:35.574798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] PATH_INFO: `/v3/users/64ac40357265482ab0c20f4c840b7721` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:35.575978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ac40357265482ab0c20f4c840b7721)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:35.577899 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:35.605357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:35.625163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ac40357265482ab0c20f4c840b7721'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:35.632118 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ac40357265482ab0c20f4c840b7721'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:35.644340 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ac40357265482ab0c20f4c840b7721'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:35.644920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ac40357265482ab0c20f4c840b7721'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:35.651670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ac40357265482ab0c20f4c840b7721'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:35.657439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1298/2593] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:35 2022] DELETE /identity/v3/users/64ac40357265482ab0c20f4c840b7721 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:35.661930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:35.662411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:35.662521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:35.680768 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:35.682518 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:35.698894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:35.741715 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1296/2594] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:35.751672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:35.753100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:35.810867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b520748-dae6-4d51-a992-7de67b1607c0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:35.814693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:35.815380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:35.815667 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:35.815942 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:35.825637 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:35.827134 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead0b3a-92f8-49a5-90f3-b4247f41bf17 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:35.837713 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1299/2595] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:36.127172 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.128984 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:36.162421 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6683245-77fd-4df3-bd5e-953d523719f8 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.168564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.169453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.169743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.170022 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] PATH_INFO: `/v3/projects/c32d8605089a46a4a902e6b3a3590bb8` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.171267 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32d8605089a46a4a902e6b3a3590bb8)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.173266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.239102 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32d8605089a46a4a902e6b3a3590bb8', 'request_id': 'req-031a521d-6b76-452a-beb1-e2eede916354'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:36.245238 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32d8605089a46a4a902e6b3a3590bb8', 'request_id': 'req-031a521d-6b76-452a-beb1-e2eede916354'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:36.256505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32d8605089a46a4a902e6b3a3590bb8', 'request_id': 'req-031a521d-6b76-452a-beb1-e2eede916354'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:36.269638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] The token cache is being invalidate because project c32d8605089a46a4a902e6b3a3590bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:36.269926 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:36.270597 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-031a521d-6b76-452a-beb1-e2eede916354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:36.272450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1297/2596] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:36 2022] DELETE /identity/v3/projects/c32d8605089a46a4a902e6b3a3590bb8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:36.278031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.278324 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.278605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.300432 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:36.301697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:36.367277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1300/2597] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:36.375939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.376228 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.376529 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.395893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:36.397167 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:36.405851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1298/2598] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:36.410195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.420137 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecca39-1f36-4a16-afcc-5045cc083d24 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.424717 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.425373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.425656 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.425936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.426658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:36.426969 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.428615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.441533 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1301/2599] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:36.445970 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.456046 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec556815-69f4-4ee2-9a36-7f93efa8e71a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.460821 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.461464 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.461746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.462018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.462843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.464772 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.489645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1299/2600] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:36.494076 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.504505 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcef490-67df-4aa4-a476-d041c0842f89 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.534840 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:36.563549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.564421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.564624 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.564904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.565845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.568214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.591550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1302/2601] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:36.593061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.603503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e6520-4f3d-4e98-9f95-23261e1aad2f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.608198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.608855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.609139 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.609415 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.610137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.611789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.622722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1300/2602] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:36.626673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.637213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba201212-18b8-49b5-b983-bcdfdc169e1a admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.641888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.642538 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.642824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.643105 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.643866 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.645462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f91767-501a-40e6-96b2-c6235f9a4a42 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.655337 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1303/2603] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:36.659475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.669621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfbb1c-c048-4a19-afe2-6cdfc5d52022 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.674114 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.675071 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.675497 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.675846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] PATH_INFO: `/v3/projects/f448852f986b4bde8229ae3bb7f62e1a/users/65e1b9339a574fa3b4c511f9a282c6a2/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.690668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f448852f986b4bde8229ae3bb7f62e1a, user_id=65e1b9339a574fa3b4c511f9a282c6a2, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.693326 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.707764 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1301/2604] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:36 2022] PUT /identity/v3/projects/f448852f986b4bde8229ae3bb7f62e1a/users/65e1b9339a574fa3b4c511f9a282c6a2/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:36.718010 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.718392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.718770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.746774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] MFA Rules not processed for user `65e1b9339a574fa3b4c511f9a282c6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:36.748062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:36.814308 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1304/2605] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:36.818818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.865675 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:36.888103 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8dcd-144b-439f-8c65-fc5799757051 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:36.893439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:36.894248 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:36.894530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:36.894811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:36.895644 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:36.898220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:36.927399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1302/2606] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:36.932430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:36.943236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56761265-bfad-4a4f-bdc5-52fdaa04e3a6 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.001633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.002315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.002602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.002882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.003605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.005558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.026299 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1305/2607] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:36 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:37.030291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.041543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6f86d-4c07-4800-8f5e-2e2d59df3e11 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.046109 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.046778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.047060 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.047359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.048079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.049665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.061609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1303/2608] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:37.065462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.075674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b50f3-a983-48e0-b7c6-de3b4d63f36f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.080259 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.081124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.081402 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.081674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] PATH_INFO: `/v3/projects/76582a4ec12f49f59026781c369c8b93/users/e4f2b628e1944040993ae269a02673bf/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.099193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] RBAC: Authorizing `identity:create_grant(project_id=76582a4ec12f49f59026781c369c8b93, user_id=e4f2b628e1944040993ae269a02673bf, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.101995 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.115188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1306/2609] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:37 2022] PUT /identity/v3/projects/76582a4ec12f49f59026781c369c8b93/users/e4f2b628e1944040993ae269a02673bf/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:37.120211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.130819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e6a39-fa34-4720-9fa3-d58fe237b6cf admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.190542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.191213 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.191516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.191787 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.192516 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.194099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b1d17-52ed-46d7-acc9-845c50ee061c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.209278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1304/2610] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:37.213188 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.223589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd43e3-0b0e-4ed3-80a0-eb524a17030b admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.228244 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.229330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.229609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.229877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] PATH_INFO: `/v3/projects/76582a4ec12f49f59026781c369c8b93/users/e4f2b628e1944040993ae269a02673bf/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.231884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76582a4ec12f49f59026781c369c8b93, user_id=e4f2b628e1944040993ae269a02673bf, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.234498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.236982 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:37.248192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1307/2611] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:37 2022] PUT /identity/v3/projects/76582a4ec12f49f59026781c369c8b93/users/e4f2b628e1944040993ae269a02673bf/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:37.258315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.258605 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.258878 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.285747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] MFA Rules not processed for user `e4f2b628e1944040993ae269a02673bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:37.286941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:37.358269 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1305/2612] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:37.369515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.371246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:37.447578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded18f-9d3b-4706-bed2-2090ac01a3f3 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.452912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.453583 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.453869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.454146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.465345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.467026 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38821e4c-db36-4ec7-91b7-20685a96ed31 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.479019 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1308/2613] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:37.872526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:37.912986 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:37.935903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd122619-bb2c-43b5-b05c-daad818f3857 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:37.940564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:37.941440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:37.941722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:37.941998 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] PATH_INFO: `/v3/users/65e1b9339a574fa3b4c511f9a282c6a2` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:37.943189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e1b9339a574fa3b4c511f9a282c6a2)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:37.945144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:37.991944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e1b9339a574fa3b4c511f9a282c6a2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:37.999119 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e1b9339a574fa3b4c511f9a282c6a2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:37.999850 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e1b9339a574fa3b4c511f9a282c6a2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.011991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e1b9339a574fa3b4c511f9a282c6a2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.018626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e1b9339a574fa3b4c511f9a282c6a2'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.024457 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1306/2614] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:37 2022] DELETE /identity/v3/users/65e1b9339a574fa3b4c511f9a282c6a2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:38.029492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733a73-751b-41b1-911c-c7012391414f None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:38.029959 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733a73-751b-41b1-911c-c7012391414f None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:38.030058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a733a73-751b-41b1-911c-c7012391414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:38.048001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1a733a73-751b-41b1-911c-c7012391414f None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:38.049192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1a733a73-751b-41b1-911c-c7012391414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:38.106963 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1309/2615] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:38.116669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:38.118061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:38.175459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4847d3-7bc5-44a8-b1b4-c4037263d4b0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:38.179165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:38.179895 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:38.180183 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:38.180461 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:38.189760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:38.191188 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:38.200790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1307/2616] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:38.463047 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733a73-751b-41b1-911c-c7012391414f None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:38.465387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1a733a73-751b-41b1-911c-c7012391414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:38.478183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a733a73-751b-41b1-911c-c7012391414f None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:38.483169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:38.484073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:38.484364 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:38.484644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] PATH_INFO: `/v3/projects/f448852f986b4bde8229ae3bb7f62e1a` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:38.485838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f448852f986b4bde8229ae3bb7f62e1a)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:38.487845 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:38.540823 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f448852f986b4bde8229ae3bb7f62e1a', 'request_id': 'req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.548747 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f448852f986b4bde8229ae3bb7f62e1a', 'request_id': 'req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.555064 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f448852f986b4bde8229ae3bb7f62e1a', 'request_id': 'req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.566634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] The token cache is being invalidate because project f448852f986b4bde8229ae3bb7f62e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:38.566920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.567642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.569851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1310/2617] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:38 2022] DELETE /identity/v3/projects/f448852f986b4bde8229ae3bb7f62e1a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:38.574346 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:38.584068 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7e45f-1533-4cde-be3d-a00fbc95f24b service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:38.659627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:38.660617 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:38.661001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:38.661376 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] PATH_INFO: `/v3/users/e4f2b628e1944040993ae269a02673bf` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:38.662945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f2b628e1944040993ae269a02673bf)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:38.665028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:38.666903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:38.720586 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f2b628e1944040993ae269a02673bf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.729885 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f2b628e1944040993ae269a02673bf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.730791 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f2b628e1944040993ae269a02673bf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.745965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f2b628e1944040993ae269a02673bf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.753506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1e7c87ce-bfef-4ac2-b99e-3074f7764c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f2b628e1944040993ae269a02673bf'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:38.770940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1308/2618] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:38 2022] DELETE /identity/v3/users/e4f2b628e1944040993ae269a02673bf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:39.052080 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.053559 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:39.067303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d148c-94fe-450e-b5bb-6a6f8985bfdc admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.089146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:39.133634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.134501 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.134807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.135109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] PATH_INFO: `/v3/projects/76582a4ec12f49f59026781c369c8b93` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.136395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=76582a4ec12f49f59026781c369c8b93)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.138492 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.187014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76582a4ec12f49f59026781c369c8b93', 'request_id': 'req-154705bd-ae19-45b7-a9ff-a2fffdd69aba'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:39.194201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76582a4ec12f49f59026781c369c8b93', 'request_id': 'req-154705bd-ae19-45b7-a9ff-a2fffdd69aba'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:39.200054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76582a4ec12f49f59026781c369c8b93', 'request_id': 'req-154705bd-ae19-45b7-a9ff-a2fffdd69aba'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:39.211617 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] The token cache is being invalidate because project 76582a4ec12f49f59026781c369c8b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:39.211900 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:39.212529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-154705bd-ae19-45b7-a9ff-a2fffdd69aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:39.214359 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1311/2619] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:39 2022] DELETE /identity/v3/projects/76582a4ec12f49f59026781c369c8b93 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:39.219176 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.219661 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.219825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.237681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:39.238905 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:39.300991 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1309/2620] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:39.310565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.310824 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.311126 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.329665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:39.330887 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:39.339086 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1312/2621] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:39.343281 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.352935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2765d3-9715-4bab-b283-7248e6881dce None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.358115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.358773 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.359057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.359365 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.360085 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:39.360394 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.361985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.373881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1310/2622] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:39.378090 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.387419 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad80207-18b8-4c66-b80a-8c874b0365f9 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.392061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.392707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.392990 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.393266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.393954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.395915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.421976 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1313/2623] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:39.426192 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.436185 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7928a-059a-4694-a9bd-6537801e28ad admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.492782 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.493456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.493742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.494020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.494732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.496681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.506105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:39.517784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1311/2624] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:39.521888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.532778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a85f4-0f7c-4d57-bfcd-7205af3b30ae admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.537461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.538371 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.538654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.538953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.539724 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.541311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.552169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1314/2625] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:39.556029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.565973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d07c4-581a-4d10-b377-9363702caa7f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.570657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.571345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.571638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.571980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.572726 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.574309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce271b35-f80b-41c2-8e2e-5bf374218bc7 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.584152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1312/2626] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:39.588169 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.598594 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740235a-19f0-4b73-b7e6-2b283f7d312f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.603443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.605140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.605429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.605708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] PATH_INFO: `/v3/projects/8a9e60aa14564ae6be6cf5d1be07137c/users/2d37e14e0618489588d2b25d7c8a787b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.620107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9e60aa14564ae6be6cf5d1be07137c, user_id=2d37e14e0618489588d2b25d7c8a787b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.622745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.635450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1315/2627] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:39 2022] PUT /identity/v3/projects/8a9e60aa14564ae6be6cf5d1be07137c/users/2d37e14e0618489588d2b25d7c8a787b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:39.645701 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.645995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.646275 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.673523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] MFA Rules not processed for user `2d37e14e0618489588d2b25d7c8a787b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:39.674733 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:39.740689 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1313/2628] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:39.745670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.800455 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:39.830867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba3d-f055-4585-87f9-19ee0d7f3d37 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.835929 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.836605 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.836888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.837166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.837874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.840401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.868931 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1316/2629] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:39.870110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.881569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3d8d4-8bed-487d-bcfb-b0dbca9e6471 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.938706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.939411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.939693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.939962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.940641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.942524 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.963692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1314/2630] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:39 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:39.967749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:39.978171 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a0bb8c-c07b-46c1-9935-f32fff7fd0ae admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:39.983397 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:39.984062 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:39.984344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:39.984620 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:39.985341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:39.986973 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:39.998243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1317/2631] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:40.002395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.012901 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf67bb-2a56-4f46-9c57-0f71fecb8d2c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.017471 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.018336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.018612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.018881 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] PATH_INFO: `/v3/projects/425e185bb6134791a693355c0bade3aa/users/99f8d2afabbd46a99ff5ad89e64c49cc/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.033304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] RBAC: Authorizing `identity:create_grant(project_id=425e185bb6134791a693355c0bade3aa, user_id=99f8d2afabbd46a99ff5ad89e64c49cc, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.035968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.049598 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1315/2632] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:40 2022] PUT /identity/v3/projects/425e185bb6134791a693355c0bade3aa/users/99f8d2afabbd46a99ff5ad89e64c49cc/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:40.053640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.064253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52ee52-7f0b-459e-b25d-4de3d7db8b86 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.118712 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.119409 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.120230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.120504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.121217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.122810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4877501-ae7e-4793-9d48-e9a1d6eecfd2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.132264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1318/2633] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:40.136526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.146746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e91df2-fb63-40be-89bb-3a410a82ecee admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.151966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.152849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.153133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.153411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] PATH_INFO: `/v3/projects/425e185bb6134791a693355c0bade3aa/users/99f8d2afabbd46a99ff5ad89e64c49cc/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.155490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425e185bb6134791a693355c0bade3aa, user_id=99f8d2afabbd46a99ff5ad89e64c49cc, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.158122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.169057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1316/2634] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:40 2022] PUT /identity/v3/projects/425e185bb6134791a693355c0bade3aa/users/99f8d2afabbd46a99ff5ad89e64c49cc/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:40.180579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.181027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.181393 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.207089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] MFA Rules not processed for user `99f8d2afabbd46a99ff5ad89e64c49cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:40.208317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:40.275574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1319/2635] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:40.288475 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.290263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:40.322237 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:40.355002 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3a3b0-b8cc-4f0b-ba02-aaa8cfe11137 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.358755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.359452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.359751 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.360027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.371439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.372566 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.383477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1317/2636] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:40.685328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.694524 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b6429-5887-49a2-8df4-760fe7426bf3 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.699783 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f8d2afabbd46a99ff5ad89e64c49cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425e185bb6134791a693355c0bade3aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.700382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.700670 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.700957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.701694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:40.702007 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.703634 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.715646 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1320/2637] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:40.719788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.729746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7383648-5d4a-4a22-b579-2259449647e3 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.734436 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f8d2afabbd46a99ff5ad89e64c49cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425e185bb6134791a693355c0bade3aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.735145 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.735587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.735870 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.736651 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.738594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.766865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1318/2638] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:40 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:40.768358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.779163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adefff-a837-4941-a074-caa7c8e92f95 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.805965 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:40.862665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f8d2afabbd46a99ff5ad89e64c49cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425e185bb6134791a693355c0bade3aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.863380 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.863668 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.863944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.864672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:40.864981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.866775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:40.881458 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1321/2639] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:40.921517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:40.932911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceebba1-6187-4c02-9fad-f73753725d70 tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:40.938978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99f8d2afabbd46a99ff5ad89e64c49cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '425e185bb6134791a693355c0bade3aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:40.940051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:40.940345 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:40.940671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] PATH_INFO: `/v3/projects/ec34b386302b4180aa823ab72c97c543` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:40.942057 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ec34b386302b4180aa823ab72c97c543)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:40.944034 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:41.013828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec34b386302b4180aa823ab72c97c543', 'request_id': 'req-50781907-d36a-4eed-8e6d-070f4e4b411f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.020509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec34b386302b4180aa823ab72c97c543', 'request_id': 'req-50781907-d36a-4eed-8e6d-070f4e4b411f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.028934 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec34b386302b4180aa823ab72c97c543', 'request_id': 'req-50781907-d36a-4eed-8e6d-070f4e4b411f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.043626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] The token cache is being invalidate because project ec34b386302b4180aa823ab72c97c543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:41.043928 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.044717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-50781907-d36a-4eed-8e6d-070f4e4b411f tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.049868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1319/2640] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:40 2022] DELETE /identity/v3/projects/ec34b386302b4180aa823ab72c97c543 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:41.261962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:41.264248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:41.337651 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5f442-33b8-4a69-8904-0c667762602a tempest-VolumeQuotaClassesTest-1972570364 tempest-VolumeQuotaClassesTest-1972570364-project-admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:41.401820 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:41.402917 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:41.403442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:41.403737 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] PATH_INFO: `/v3/users/2d37e14e0618489588d2b25d7c8a787b` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:41.405206 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d37e14e0618489588d2b25d7c8a787b)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:41.407201 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:41.460264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d37e14e0618489588d2b25d7c8a787b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.468379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d37e14e0618489588d2b25d7c8a787b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.483130 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d37e14e0618489588d2b25d7c8a787b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.483853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d37e14e0618489588d2b25d7c8a787b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.485045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:41.492979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d37e14e0618489588d2b25d7c8a787b'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:41.500113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1322/2641] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:41 2022] DELETE /identity/v3/users/2d37e14e0618489588d2b25d7c8a787b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:41.504645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:41.505115 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:41.505386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:41.522408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:41.523770 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:41.589229 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1320/2642] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:41.600502 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:41.601714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:41.669781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd95b73-b35c-4d9d-a6ff-142a672a6fcc admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:41.673574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:41.674234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:41.674517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:41.674792 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:41.685545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:41.687497 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:41.697388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1323/2643] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:41.974243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:41.975766 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:41.993381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d5691-4574-4680-8107-8b18b09f2352 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:41.999160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:41.999956 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.000277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.000481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] PATH_INFO: `/v3/projects/8a9e60aa14564ae6be6cf5d1be07137c` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.001667 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9e60aa14564ae6be6cf5d1be07137c)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:42.003626 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:42.035217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:42.054116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9e60aa14564ae6be6cf5d1be07137c', 'request_id': 'req-d6c3313e-9e82-4770-9fc9-520ca9bde207'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.060288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9e60aa14564ae6be6cf5d1be07137c', 'request_id': 'req-d6c3313e-9e82-4770-9fc9-520ca9bde207'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.067508 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9e60aa14564ae6be6cf5d1be07137c', 'request_id': 'req-d6c3313e-9e82-4770-9fc9-520ca9bde207'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.078864 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] The token cache is being invalidate because project 8a9e60aa14564ae6be6cf5d1be07137c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:42.079144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.079792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.081313 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1321/2644] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:41 2022] DELETE /identity/v3/projects/8a9e60aa14564ae6be6cf5d1be07137c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:42.085330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:42.096271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742169fd-2fc9-4c0c-85b9-fe30a50f9b70 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:42.152196 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:42.153070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.153349 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.153619 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] PATH_INFO: `/v3/users/99f8d2afabbd46a99ff5ad89e64c49cc` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.154849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f8d2afabbd46a99ff5ad89e64c49cc)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:42.156771 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:42.211316 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f8d2afabbd46a99ff5ad89e64c49cc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.220858 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f8d2afabbd46a99ff5ad89e64c49cc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.236288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f8d2afabbd46a99ff5ad89e64c49cc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.237255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f8d2afabbd46a99ff5ad89e64c49cc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.244554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-9840d572-80fb-4376-a41d-baf0e17748c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f8d2afabbd46a99ff5ad89e64c49cc'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.251388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1324/2645] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:42 2022] DELETE /identity/v3/users/99f8d2afabbd46a99ff5ad89e64c49cc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:42.542592 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:42.544552 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:42.560154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3313e-9e82-4770-9fc9-520ca9bde207 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:42.621593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:42.622452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.622732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.623003 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] PATH_INFO: `/v3/projects/425e185bb6134791a693355c0bade3aa` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.624209 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] RBAC: Authorizing `identity:delete_project(project_id=425e185bb6134791a693355c0bade3aa)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:42.626106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:42.672243 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425e185bb6134791a693355c0bade3aa', 'request_id': 'req-1188a9e4-3e83-47f1-81c0-fa40c524d28f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.677660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425e185bb6134791a693355c0bade3aa', 'request_id': 'req-1188a9e4-3e83-47f1-81c0-fa40c524d28f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.684650 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425e185bb6134791a693355c0bade3aa', 'request_id': 'req-1188a9e4-3e83-47f1-81c0-fa40c524d28f'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.695587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] The token cache is being invalidate because project 425e185bb6134791a693355c0bade3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:42.695899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.696525 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-1188a9e4-3e83-47f1-81c0-fa40c524d28f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:42.698138 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1322/2646] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:42 2022] DELETE /identity/v3/projects/425e185bb6134791a693355c0bade3aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:42.704353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.704833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.704940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.723444 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:42.724649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:42.730609 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:42.791163 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1325/2647] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:42.799871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.800194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.800851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.822182 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:42.823549 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:42.835233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1323/2648] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:42.837180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:42.852815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af434e0-536b-4fa3-b616-3f27a672d882 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:42.858207 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:42.858933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.859372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.859517 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.860243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:42.860602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:42.862213 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:42.882542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1326/2649] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:42.884082 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:42.902505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9eb9e-8a34-4e0a-ac7e-c011229be730 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:42.907830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:42.908505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:42.908792 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:42.909073 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:42.909780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:42.911723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:42.951858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1324/2650] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:42 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:42.953035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:42.973430 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9a8fb-7a34-407b-8a04-13a2c958904d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:43.030691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:43.031406 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.031698 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.031999 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.032699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:43.034657 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:43.055691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1327/2651] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:42 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:43.060329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:43.070737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b150db-9299-4517-86f6-c17ab4c2d20c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:43.075600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:43.076263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.076547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.076828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.077660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:43.079339 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:43.089753 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1325/2652] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:43.093766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:43.104462 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d4a9d-3aed-4b47-b097-ac4ce51ecbc4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:43.109315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:43.109981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.110265 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.110542 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.111267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:43.112909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a89054-1d41-4b9e-96f8-bbb314750b99 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:43.123915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1328/2653] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:42:43.128121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:43.137338 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:43.141855 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de1c7d-7c89-46bd-a5ff-6b812f8885fa admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:43.152564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:43.153476 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.153761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.154037 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] PATH_INFO: `/v3/projects/97744810d70f406b8d3a2cab1b165042/users/36e61cae0668412aae079d10941c2835/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.173562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97744810d70f406b8d3a2cab1b165042, user_id=36e61cae0668412aae079d10941c2835, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:43.176406 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:43.191486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1326/2654] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:43 2022] PUT /identity/v3/projects/97744810d70f406b8d3a2cab1b165042/users/36e61cae0668412aae079d10941c2835/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:43.201905 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.202199 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.202551 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.232180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] MFA Rules not processed for user `36e61cae0668412aae079d10941c2835`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:43.233572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:43.301067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1329/2655] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:43.312997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:43.315123 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:43.385723 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f36ef-28c3-43eb-8ce4-a066f4f73d3e admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:43.391008 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:43.392348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:43.392876 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:43.393342 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:43.403692 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:43.405385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfb1fa-2e6d-4857-b668-dc7b31770a1a service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:43.417210 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1327/2656] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:44.844071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:44.845785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:44.859933 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:44.925079 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9f90-c49a-422f-9b7f-86e7b0ee4966 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:44.931119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:44.932416 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:44.933108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:44.933443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:44.935269 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:45.009881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:45.011439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:45.021078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1330/2657] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:45.132606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20395e55-b818-4492-b273-c4c770b74d70 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:45.133132 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20395e55-b818-4492-b273-c4c770b74d70 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:45.133875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20395e55-b818-4492-b273-c4c770b74d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:45.151569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-20395e55-b818-4492-b273-c4c770b74d70 None None] MFA Rules not processed for user `354e3e9955a74c1a86b7e4e6c9bbd512`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:45.152960 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-20395e55-b818-4492-b273-c4c770b74d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:45.212856 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1328/2658] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:46.001218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:46.075862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a02b4-1a80-46f9-bfc0-bf21d6e60dea service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:46.081100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:46.082141 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:46.082521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:46.082857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] PATH_INFO: `/v3/users/36e61cae0668412aae079d10941c2835` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:46.084252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e61cae0668412aae079d10941c2835)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:46.086461 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:46.143673 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e61cae0668412aae079d10941c2835'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:46.154204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e61cae0668412aae079d10941c2835'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:46.159493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:46.175946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e61cae0668412aae079d10941c2835'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:46.176720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e61cae0668412aae079d10941c2835'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:46.185590 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e61cae0668412aae079d10941c2835'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:46.192087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1331/2659] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:42:46 2022] DELETE /identity/v3/users/36e61cae0668412aae079d10941c2835 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:46.197225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:46.197784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:46.198140 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:46.224620 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:46.228641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:46.303722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1329/2660] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:42:46.316521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:46.319410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:46.384749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1002-7368-4184-952a-58ec0992f155 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:46.389688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:46.390373 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:46.390663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:46.390934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:46.402972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:46.404459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1bbba-011f-4311-a516-bf0259cd0fe2 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:46.415096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1332/2661] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:46.889882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:46.891380 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:46.907797 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b8a81-78fe-4e30-ae94-fd663b0d586d None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:46.914830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:46.917153 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:46.917381 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:46.917759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] PATH_INFO: `/v3/projects/97744810d70f406b8d3a2cab1b165042` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:46.919843 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97744810d70f406b8d3a2cab1b165042)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:46.924129 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:46.967000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:46.995741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97744810d70f406b8d3a2cab1b165042', 'request_id': 'req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:47.001316 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97744810d70f406b8d3a2cab1b165042', 'request_id': 'req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:47.009019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97744810d70f406b8d3a2cab1b165042', 'request_id': 'req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:47.021924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] The token cache is being invalidate because project 97744810d70f406b8d3a2cab1b165042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:42:47.022211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:47.022887 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-5cdf6987-6ba2-4cdd-b9c5-f9dd6b535299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:42:47.024492 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1330/2662] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:42:46 2022] DELETE /identity/v3/projects/97744810d70f406b8d3a2cab1b165042 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:42:47.030894 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.031431 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.031536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.050291 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:47.051248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:47.127313 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1333/2663] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:47.136202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.136486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.136762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.156645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:47.158137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:47.167395 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1331/2664] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:47.179618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.190437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db7963-018c-444d-a261-58abc4be28e1 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.195097 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.195801 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.196284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.196565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.197298 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:42:47.198214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.199977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.215381 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1334/2665] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:42:47.219875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.230134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db538c88-4ba5-48fc-97d4-bb886dd7141a None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.236352 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.237027 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.237304 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.237585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.238262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.240220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.278841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1332/2666] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:42:47 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:42:47.281790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.296439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b4d11-1847-4823-b5cd-47f9f46bcdef admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.390587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.391305 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.391600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.391881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.392598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.398749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.416786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:47.443476 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1335/2667] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:42:47 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:42:47.450892 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.464989 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa02a-111d-4681-b88b-02b6e59933d9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.469999 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.470832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.471116 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.471453 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.472201 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.473799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.486441 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1333/2668] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:47.490429 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.500834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cb881-c6ea-4a1f-a5bc-1f7ec8649cb4 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.505592 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.506293 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.506578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.506856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.507612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.509215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b64dec-594a-41ff-aaec-21f2885a38fc admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.520379 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1336/2669] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:42:47.524400 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.535521 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109d703-e215-4458-a4c6-bd4c67478d0f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.540443 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.541378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.541688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.541968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] PATH_INFO: `/v3/projects/8c863dfe96b34ab998cbd8e4637b1610/users/3686d32083404712bd6dd4f204432c39/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.556632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c863dfe96b34ab998cbd8e4637b1610, user_id=3686d32083404712bd6dd4f204432c39, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.559530 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.574331 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1334/2670] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:42:47 2022] PUT /identity/v3/projects/8c863dfe96b34ab998cbd8e4637b1610/users/3686d32083404712bd6dd4f204432c39/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:42:47.584152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca602b-1ca9-4394-b60b-b761e0e3b9ba None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.584544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca602b-1ca9-4394-b60b-b761e0e3b9ba None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.584895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca602b-1ca9-4394-b60b-b761e0e3b9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.612915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-46ca602b-1ca9-4394-b60b-b761e0e3b9ba None None] MFA Rules not processed for user `3686d32083404712bd6dd4f204432c39`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:42:47.614167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-46ca602b-1ca9-4394-b60b-b761e0e3b9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:47.687336 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1337/2671] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:42:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:42:47.701197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:42:47.704968 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:42:47.773673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64ae8f-e8b6-4c6c-bb55-4ee960a39bff admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:42:47.780017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2e6dffba870453fab836e0ccece7708', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:42:47.780703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:42:47.780995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:42:47.781273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:42:47.790937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:42:47.792638 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:42:47.802950 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1335/2672] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:42:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:42:57.683081 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=39 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:42:57.798358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:08.030133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:08.030633 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:08.030725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:08.050067 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:08.051271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:08.107153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1338/2673] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Jul 20 09:43:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:08.130632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:08.132130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:08.219028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f12e56-970a-49a3-b183-278e2af542a3 service glance] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:08.224386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:08.225040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:08.225323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:08.225761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:08.234706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:08.236315 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:08.246925 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1336/2674] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:43:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:43:09.482766 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:09.485356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:09.572342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aba971-758b-459b-aa1f-9b240db29a64 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:09.576528 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:09.577262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:09.577549 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:09.577836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:09.579710 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:09.653016 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:09.654388 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:09.665608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1339/2675] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:13.483007 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:13.484600 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:13.520759 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d5c22-2965-4f84-af86-90282e2e8745 service swift] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:13.526036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:13.527067 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:13.527575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:13.528054 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] PATH_INFO: `/v3/users/a37198e6d4424a51800655d3bd0a3ee6` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:13.529460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a37198e6d4424a51800655d3bd0a3ee6)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:13.531413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:13.595691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a37198e6d4424a51800655d3bd0a3ee6'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:13.604434 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a37198e6d4424a51800655d3bd0a3ee6'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:13.606043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a37198e6d4424a51800655d3bd0a3ee6'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:13.623861 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a37198e6d4424a51800655d3bd0a3ee6'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:13.636432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-766d8549-4081-4cd9-8a6a-9dae1714b199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a37198e6d4424a51800655d3bd0a3ee6'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:13.648020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1337/2676] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:43:13 2022] DELETE /identity/v3/users/a37198e6d4424a51800655d3bd0a3ee6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:13.868193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:13.878744 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5120173-5822-460d-9ef6-135a0d08968e service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:13.959912 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:13.969944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:13.970888 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:13.971401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:13.971681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] PATH_INFO: `/v3/projects/07eb1473648d490b90a1e7debc45d57d` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:13.973005 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07eb1473648d490b90a1e7debc45d57d)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:13.975095 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.033332 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07eb1473648d490b90a1e7debc45d57d', 'request_id': 'req-6a671529-d8e5-4ea5-a2eb-82cebe95192e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:14.042791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07eb1473648d490b90a1e7debc45d57d', 'request_id': 'req-6a671529-d8e5-4ea5-a2eb-82cebe95192e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:14.049951 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07eb1473648d490b90a1e7debc45d57d', 'request_id': 'req-6a671529-d8e5-4ea5-a2eb-82cebe95192e'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:14.068181 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] The token cache is being invalidate because project 07eb1473648d490b90a1e7debc45d57d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:43:14.068522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:14.069211 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-6a671529-d8e5-4ea5-a2eb-82cebe95192e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:14.071071 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1340/2677] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:43:13 2022] DELETE /identity/v3/projects/07eb1473648d490b90a1e7debc45d57d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:14.077673 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.078446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.079029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.098839 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:14.100126 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:14.185664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1338/2678] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:14.194100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.194479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.194828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.215903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:14.217158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:14.225490 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1341/2679] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:14.229637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.235154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:14.240660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c030877-7d22-4610-b01a-c7e685bb1684 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.251106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.251805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.252090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.252363 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.253088 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:43:14.253513 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.255122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.270033 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1339/2680] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:43:14.274643 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.289769 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546e875-2a62-439d-8b1d-1bdf5c1c4fbd None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.290674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.291478 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.291903 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.292178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.293192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.295127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.327685 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1342/2681] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:43:14 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:14.331860 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.343035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8bf6b-62ea-4107-9412-eebfb5fe5335 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.412490 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.414079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.414575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.414936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.415810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.417868 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.446762 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1340/2682] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:43:14 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:14.451531 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.464927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b610929-b8da-4e9f-bb3b-eb71f73dc4b5 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.471003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.471745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.472032 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.472310 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.473122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.474825 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.490011 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1343/2683] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:14.490966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.501858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649ea16-6830-465a-9b3a-4782f707a3ef admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.509814 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.510514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.510802 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.511080 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.512575 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.514420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507794e0-981d-4202-a781-a022f25a96a0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.526455 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1341/2684] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:43:14.531798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.544772 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdc9de-b6cd-4b0f-b065-d03567a11fb0 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.549574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.550570 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.550991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.551274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] PATH_INFO: `/v3/projects/a1fe308758354ddfb050def8345355ff/users/ae56ce45fd2d4acab2c1959a45dfe00b/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.571554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fe308758354ddfb050def8345355ff, user_id=ae56ce45fd2d4acab2c1959a45dfe00b, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.575317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.593979 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1344/2685] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:14 2022] PUT /identity/v3/projects/a1fe308758354ddfb050def8345355ff/users/ae56ce45fd2d4acab2c1959a45dfe00b/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:14.599688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.600096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.600482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.630537 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:14.631854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:14.718722 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1342/2686] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:14.730189 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:14.732640 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:14.828863 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac7a1a-cc6c-4874-8aa0-d3bac51ee763 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:14.834788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:14.836271 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:14.836911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:14.837347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:14.849568 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:14.851387 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf71e9-1387-4314-8177-1dbe6c3b5963 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:14.863178 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1345/2687] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:18.319144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:18.321631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:18.350255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e73247-d7ce-4368-951e-c1293b38f5db None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:18.364065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:18.365660 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:18.366498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:18.366940 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] PATH_INFO: `/v3/users/3686d32083404712bd6dd4f204432c39` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:18.368448 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3686d32083404712bd6dd4f204432c39)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:18.374419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:18.485456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3686d32083404712bd6dd4f204432c39'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:18.495717 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3686d32083404712bd6dd4f204432c39'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:18.496835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3686d32083404712bd6dd4f204432c39'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:18.517440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:18.527610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3686d32083404712bd6dd4f204432c39'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:18.551266 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3686d32083404712bd6dd4f204432c39'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:18.568995 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1343/2688] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:43:18 2022] DELETE /identity/v3/users/3686d32083404712bd6dd4f204432c39 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:18.573342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:18.575229 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:18.576264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:18.612140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:18.613867 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:18.659437 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:18.727408 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1346/2689] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:18.743891 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:18.746336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:18.870462 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f093a4-9b46-4781-bd13-b6fb729f94f9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:18.884017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:18.885370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:18.885914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:18.887025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:18.902417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:18.904656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ee3dd-a711-4b5c-b4ab-f8d9321b5883 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:18.931681 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1344/2690] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:19.488598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:19.490944 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:19.522075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87452526-09a7-44b1-8d00-3bab4402d76b None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:19.531794 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:19.535166 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:19.536150 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:19.536443 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] PATH_INFO: `/v3/projects/8c863dfe96b34ab998cbd8e4637b1610` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:19.537798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c863dfe96b34ab998cbd8e4637b1610)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:19.541138 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:19.679135 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c863dfe96b34ab998cbd8e4637b1610', 'request_id': 'req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:19.696718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c863dfe96b34ab998cbd8e4637b1610', 'request_id': 'req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:19.711716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c863dfe96b34ab998cbd8e4637b1610', 'request_id': 'req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:19.734957 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] The token cache is being invalidate because project 8c863dfe96b34ab998cbd8e4637b1610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:43:19.735911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:19.741328 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae29f6b2-b7c7-47ce-9051-87519a99f8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:19.749063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1347/2691] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:43:19 2022] DELETE /identity/v3/projects/8c863dfe96b34ab998cbd8e4637b1610 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:19.751459 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:19.751942 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:19.752285 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:19.787957 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:19.790268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:19.911688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:19.911978 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:19.912785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:19.922398 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1345/2692] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:19.947015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:19.950793 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:19.969499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1348/2693] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:19.972577 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:19.992828 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e98b3f-b00d-4917-a771-775d073af639 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:19.998178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:19.999136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:19.999258 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:19.999873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.000277 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:43:20.000587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.002202 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.023113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1346/2694] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:43:20.024319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.040687 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f8ff8-2566-459d-9377-d5d9a1ea0973 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.055222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.056162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.056627 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.057091 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.058058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.062795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.132350 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1349/2695] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:43:20 2022] POST /identity/v3/projects => generated 397 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:20.136919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.153949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f243f9-2041-4970-abb0-aa120299a966 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.271973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.272961 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.273847 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.274288 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.275156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.277883 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.297867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:20.312670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1347/2696] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:43:20 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:20.314215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.331923 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28278d-52bc-4d9e-bb77-0387a343fa05 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.337719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.338410 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.338694 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.339038 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.340446 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.342809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.359785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1350/2697] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:20.361026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.378470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d72c76-96c5-49f2-ac28-f104b43ce302 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.387875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.390528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.391064 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.391564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.393025 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.395043 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27725d04-91cc-47d7-b65d-35226ce2ba11 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.412915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.413166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1348/2698] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:20.434767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.435402 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.435873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.446645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20392c-0772-4ec0-a7cd-c10e0e814f86 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.457434 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.458615 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.459001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.459401 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] PATH_INFO: `/v3/projects/e5e0db7dca5949a28baffdc3a058cf14/users/f2ed3f407ff348d19a9fd151bbe71873/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.477156 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] MFA Rules not processed for user `ae56ce45fd2d4acab2c1959a45dfe00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:20.478480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:20.489447 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e0db7dca5949a28baffdc3a058cf14, user_id=f2ed3f407ff348d19a9fd151bbe71873, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.492554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdd115-be2e-4ca6-9303-052065341f20 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.524153 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1351/2699] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:20 2022] PUT /identity/v3/projects/e5e0db7dca5949a28baffdc3a058cf14/users/f2ed3f407ff348d19a9fd151bbe71873/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:20.525678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.526209 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.526499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.559493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] MFA Rules not processed for user `f2ed3f407ff348d19a9fd151bbe71873`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:20.560895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:20.577177 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1349/2700] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:20.594591 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.598181 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:20.628264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:20.704415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1352/2701] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:20.719926 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:20.721770 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:20.741927 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c537a-bcf7-493d-a4bd-79dae4f5df83 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.843636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f8f0c-d904-4dae-be5a-99aede7bad3a None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:20.854636 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.855600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.860383 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.860714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.874113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.876106 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ade02-0dc0-4779-846f-da2214a4eac1 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:20.893746 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:20.895049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:20.895543 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:20.896386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:20.906653 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1353/2702] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:43:20.913924 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=100 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:20.986106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:20.988564 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3c2e6-a730-4ac7-9aa4-f8f49ed431f4 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:21.015245 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1350/2703] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:22.769804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:22.770589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:22.770589 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:22.795645 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] MFA Rules not processed for user `ae56ce45fd2d4acab2c1959a45dfe00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:22.797749 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:22.811433 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1354/2704] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:27.626793 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:27.627264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:27.627391 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:27.644282 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] MFA Rules not processed for user `1ab2924b8a9d41a7a6b74a73a7b809bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:27.645618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:27.701985 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1351/2705] 10.222.0.18 () {58 vars in 967 bytes} [Wed Jul 20 09:43:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:27.715314 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:27.717736 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:27.806179 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37405be9-2056-494f-9443-9600f95ebb55 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:27.811560 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '826539d8549b40ddacbe997a519691ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:27.812233 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:27.812518 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:27.812795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:27.821362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:27.823014 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d9fbe-97c9-4c5d-a4b5-7d0d4dc7cc27 service swift] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:27.831877 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1355/2706] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Jul 20 09:43:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:43:29.605065 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:29.668904 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b533a9-7fcc-446a-8660-dcdb877a0dfe None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:29.673440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:29.674416 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:29.674840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:29.675205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] PATH_INFO: `/v3/users/f2ed3f407ff348d19a9fd151bbe71873` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:29.676481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ed3f407ff348d19a9fd151bbe71873)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:29.678450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:29.726272 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ed3f407ff348d19a9fd151bbe71873'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.735704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ed3f407ff348d19a9fd151bbe71873'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.735994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ed3f407ff348d19a9fd151bbe71873'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.741997 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:29.751829 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ed3f407ff348d19a9fd151bbe71873'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.759632 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ed3f407ff348d19a9fd151bbe71873'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.765805 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1352/2707] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:43:29 2022] DELETE /identity/v3/users/f2ed3f407ff348d19a9fd151bbe71873 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:29.770192 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:29.770665 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:29.770856 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:29.790435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:29.791677 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:29.844723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=100 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:29.855466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1356/2708] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:29.861310 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:29.862838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:29.866729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:29.868578 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:29.879039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6c7e-6d18-4221-ab49-ac83a83546ac admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:29.884168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:29.885262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:29.885760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:29.886958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] PATH_INFO: `/v3/users/354e3e9955a74c1a86b7e4e6c9bbd512` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:29.888447 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=354e3e9955a74c1a86b7e4e6c9bbd512)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:29.899642 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:29.939705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce75d09-24a4-4fc2-9915-d8e45f3ba391 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:29.946347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:29.947008 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:29.947311 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:29.947644 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:29.955383 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354e3e9955a74c1a86b7e4e6c9bbd512'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.958223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:29.959720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:29.963939 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354e3e9955a74c1a86b7e4e6c9bbd512'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.965160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354e3e9955a74c1a86b7e4e6c9bbd512'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.973711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1357/2709] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:43:29.980374 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354e3e9955a74c1a86b7e4e6c9bbd512'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.989329 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354e3e9955a74c1a86b7e4e6c9bbd512'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:29.996738 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1353/2710] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:43:29 2022] DELETE /identity/v3/users/354e3e9955a74c1a86b7e4e6c9bbd512 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:30.321892 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.332063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40124a6-9f46-4489-9a65-9a98c9736e08 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.399791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.402267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.402267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.402267 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] PATH_INFO: `/v3/projects/4f8dfa60d69d45c29013dbbe6c1dd226` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.403330 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8dfa60d69d45c29013dbbe6c1dd226)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.405243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.408499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.410142 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:30.425211 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b0ca7-0309-40f2-894e-434b0d0fd5eb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.429996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.430846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.431194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.431419 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] PATH_INFO: `/v3/projects/e5e0db7dca5949a28baffdc3a058cf14` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.445779 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e0db7dca5949a28baffdc3a058cf14)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.447965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.462846 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8dfa60d69d45c29013dbbe6c1dd226', 'request_id': 'req-64be5101-4205-4ead-b41f-eedb0ad225e3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.471172 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8dfa60d69d45c29013dbbe6c1dd226', 'request_id': 'req-64be5101-4205-4ead-b41f-eedb0ad225e3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.477100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8dfa60d69d45c29013dbbe6c1dd226', 'request_id': 'req-64be5101-4205-4ead-b41f-eedb0ad225e3'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.490066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] The token cache is being invalidate because project 4f8dfa60d69d45c29013dbbe6c1dd226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:43:30.490351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.490986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-64be5101-4205-4ead-b41f-eedb0ad225e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.492671 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1358/2711] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:43:30 2022] DELETE /identity/v3/projects/4f8dfa60d69d45c29013dbbe6c1dd226 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:30.497842 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9291a6-c77f-4788-8df2-b5c884f0f051 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.498042 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e0db7dca5949a28baffdc3a058cf14', 'request_id': 'req-59cd571f-73ce-487b-b000-eab224a43260'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.498155 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9291a6-c77f-4788-8df2-b5c884f0f051 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.498400 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9291a6-c77f-4788-8df2-b5c884f0f051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.503657 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e0db7dca5949a28baffdc3a058cf14', 'request_id': 'req-59cd571f-73ce-487b-b000-eab224a43260'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.513432 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e0db7dca5949a28baffdc3a058cf14', 'request_id': 'req-59cd571f-73ce-487b-b000-eab224a43260'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.518883 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3f9291a6-c77f-4788-8df2-b5c884f0f051 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:30.520140 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3f9291a6-c77f-4788-8df2-b5c884f0f051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:30.524682 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] The token cache is being invalidate because project e5e0db7dca5949a28baffdc3a058cf14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:43:30.524973 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.525606 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-59cd571f-73ce-487b-b000-eab224a43260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:30.527341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1354/2712] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:43:30 2022] DELETE /identity/v3/projects/e5e0db7dca5949a28baffdc3a058cf14 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:30.532075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.532379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.532640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.551990 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:30.553131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:30.553292 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:30.588096 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:30.588898 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1359/2713] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:30.598225 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8f92-d47a-44f9-ae12-8a0c2be7b326 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.598525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8f92-d47a-44f9-ae12-8a0c2be7b326 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.598809 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e8f92-d47a-44f9-ae12-8a0c2be7b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.618261 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-a16e8f92-d47a-44f9-ae12-8a0c2be7b326 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:30.619511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a16e8f92-d47a-44f9-ae12-8a0c2be7b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:30.626300 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1355/2714] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:30.628675 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1360/2715] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:30.632930 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.634284 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.634574 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.634849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.645264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2430-f158-403f-8d8b-716310ee0230 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.652468 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.653514 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.653799 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.654075 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.654840 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:43:30.655233 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.656759 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:30.657198 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.657996 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:30.670493 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1361/2716] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:30.670691 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1356/2717] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:43:30.671526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.674979 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.681000 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5376c-4e3b-4c54-b59f-6140373c9327 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.688264 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d182318-6bd6-4ae5-8871-4baff102cf81 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.689510 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.690454 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.690819 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.691096 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] PATH_INFO: `/v3/domains` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.691949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80185) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:43:30.692366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.694173 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.694303 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.694880 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.695095 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.695399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.696234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.698108 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.708242 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1362/2718] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:43:30.712557 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.726057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6a5df-5f40-4682-b2d9-3935bccf3197 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.727967 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1357/2719] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:43:30.734217 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.792909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.793813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.794220 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.794597 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.795608 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.797545 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.808648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31660fc9-0237-459c-a73b-dcc9a5275318 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.813815 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.814703 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.815023 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.815569 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.816299 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.818612 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.824756 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1363/2720] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:30.833781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.844496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1358/2721] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:30.848503 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.858212 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597498f-159c-46b0-bc1d-fc89394ed61f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.903522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbfc53-5eaa-41ae-9a94-ebd8514d7522 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.908262 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.908960 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.909237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.909515 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] PATH_INFO: `/v3/users` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.910205 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.912146 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.922372 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.923048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.923378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.923670 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.924580 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.926328 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.937054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1364/2722] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:43:30 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:43:30.938107 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.943982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1359/2723] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:43:30.944981 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.948908 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0a946-71ac-4544-a231-f8e79ddf57dd admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.953741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.954407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.954692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.954967 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.955733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.956105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea8b5a-ae70-45c9-ae18-f53a6041cd12 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.957348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.960922 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.961571 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.961853 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.962121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.962830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.965412 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.972059 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1365/2724] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:30.973218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.977936 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1360/2725] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:30.979175 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:30.983582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f42df9-1a35-4efc-8dc4-ac6e8f7073cb admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.988227 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.988869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.989113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b86cd-01d4-47bc-927e-3f996aff4a66 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:30.989224 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.989415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.990162 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:30.991774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:30.994120 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:30.994996 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:30.995358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:30.995874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] PATH_INFO: `/v3/projects/ad31f1468e9c41eaa4bfec667e605b68/users/7a5753eb9cad42c1a40ab505623cda8d/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:30.997598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:31.006584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1366/2726] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:31.007571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:31.008309 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:31.013582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad31f1468e9c41eaa4bfec667e605b68, user_id=7a5753eb9cad42c1a40ab505623cda8d, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:31.016411 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3970c-9add-48a2-b5cc-c28a392e1316 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:31.018572 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a357d-a2ef-4171-a921-d57144d94e1e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:31.023966 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:31.024874 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:31.025234 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:31.025588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] PATH_INFO: `/v3/projects/42a88572ee8443d4a51617c04ca2f416/users/ac343929fc8542439bc627a1f445b1dd/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:31.031449 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1361/2727] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:30 2022] PUT /identity/v3/projects/ad31f1468e9c41eaa4bfec667e605b68/users/7a5753eb9cad42c1a40ab505623cda8d/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:31.041704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:31.041886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a88572ee8443d4a51617c04ca2f416, user_id=ac343929fc8542439bc627a1f445b1dd, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:31.041994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:31.042307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:31.045289 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809d0de-7237-42fa-88b6-07464d39ad2b admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:31.058875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1367/2728] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:31 2022] PUT /identity/v3/projects/42a88572ee8443d4a51617c04ca2f416/users/ac343929fc8542439bc627a1f445b1dd/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:31.063934 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:31.064425 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:31.064707 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:31.070168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] MFA Rules not processed for user `7a5753eb9cad42c1a40ab505623cda8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:31.071437 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:31.082550 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:31.083773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:31.135422 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1362/2729] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:31.148285 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1368/2730] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:43:31.148496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:31.148496 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:31.155555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:31.156935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:31.206360 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f7ccd-5318-43f9-876a-83a0f993c7ac None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:31.209993 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:31.210665 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:31.210941 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:31.211215 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:31.220951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:31.221132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643007-3df4-4c60-ae0c-83a07fc2f059 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:31.222384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66d253-1a8f-4414-b4e9-c4c09da43e22 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:31.224879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:31.225529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:31.225848 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:31.226249 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:31.231949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1363/2731] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:43:31.236113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:31.237602 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:31.247312 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1369/2732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:43:35.725875 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:35.726337 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:35.726431 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:35.753668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] MFA Rules not processed for user `ac343929fc8542439bc627a1f445b1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:35.755049 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:35.834013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1364/2733] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:35.837688 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:35.848217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d604a4-8288-4bdd-a212-252ceaece96b service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:35.853210 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:35.853946 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:35.854237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:35.854561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:35.855348 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:35.857471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:35.897561 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1370/2734] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:43:35 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:35.902036 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:35.915242 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95be67-f1d9-4853-a8ff-7445d0589179 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:35.928118 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:36.050769 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:36.051572 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:36.051921 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:36.052122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:36.052835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:36.062011 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:36.087324 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1365/2735] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:43:35 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:43:36.090035 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:36.101600 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b1e3-f0b8-40ce-b9d7-5950d8773cb2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:36.107591 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:36.108711 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:36.109013 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:36.109286 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:36.110012 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:36.111650 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:36.126836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1371/2736] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:36.128375 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:36.157731 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd17ae-d399-4b6f-aad1-be05cf286cf4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:36.163784 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:36.164910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:36.165195 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:36.165473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] PATH_INFO: `/v3/projects/cb20371807744e2cace9f80c282e7004/users/5e79fedff78d459d8578a3ae093fd8aa/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:36.184923 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb20371807744e2cace9f80c282e7004, user_id=5e79fedff78d459d8578a3ae093fd8aa, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:36.187639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:36.202528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1366/2737] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:36 2022] PUT /identity/v3/projects/cb20371807744e2cace9f80c282e7004/users/5e79fedff78d459d8578a3ae093fd8aa/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:36.206720 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:36.233795 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3c91b-25a4-4a50-b1ef-1b5f0c2aff78 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:36.282696 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:36.298598 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:36.299399 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:36.299758 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:36.300104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:36.300901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:36.302950 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b69ab0-2e95-425d-b89f-e2b5fd0f8296 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:36.314601 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1372/2738] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:43:36.318590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:36.332013 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df7d0b-a083-4727-a692-5d3482c4526a admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:36.337029 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:36.337933 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:36.338222 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:36.338504 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] PATH_INFO: `/v3/projects/cb20371807744e2cace9f80c282e7004/users/5e79fedff78d459d8578a3ae093fd8aa/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:36.341157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb20371807744e2cace9f80c282e7004, user_id=5e79fedff78d459d8578a3ae093fd8aa, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:36.343886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:36.355669 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1367/2739] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:43:36 2022] PUT /identity/v3/projects/cb20371807744e2cace9f80c282e7004/users/5e79fedff78d459d8578a3ae093fd8aa/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:43:42.025788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:42.026704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:42.027161 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:42.075031 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] MFA Rules not processed for user `5e79fedff78d459d8578a3ae093fd8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:42.076255 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:42.188117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1373/2740] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:42.202094 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:42.204463 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:42.345585 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:42.358656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cbf9c-e17e-40b7-b401-ffabf5cf15c0 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:42.364427 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:42.365107 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:42.365414 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:42.365698 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:42.421556 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:42.423146 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558934df-0ee0-4901-9eaa-22d7670112ca service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:42.432020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1368/2741] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:42.622692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:42.711652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e5ac-4169-437b-a7e8-fa829dc2d0ca None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:42.717113 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:42.717991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:42.718282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:42.718567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] PATH_INFO: `/v3/users/7a5753eb9cad42c1a40ab505623cda8d` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:42.719799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5753eb9cad42c1a40ab505623cda8d)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:42.722054 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:42.748078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:42.780947 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5753eb9cad42c1a40ab505623cda8d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:42.789843 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5753eb9cad42c1a40ab505623cda8d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:42.805642 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5753eb9cad42c1a40ab505623cda8d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:42.806372 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5753eb9cad42c1a40ab505623cda8d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:42.815204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5753eb9cad42c1a40ab505623cda8d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:42.822124 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1374/2742] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:43:42 2022] DELETE /identity/v3/users/7a5753eb9cad42c1a40ab505623cda8d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:42.827351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:42.827639 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:42.827917 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:42.921203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:42.922403 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:43.003343 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1369/2743] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:43.016459 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:43.018084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:43.095148 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cab2b9-b562-4d1a-9f1a-8cc9a358058f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:43.101111 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:43.101881 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:43.102239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:43.102587 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:43.115879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:43.117465 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:43.130277 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1375/2744] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:43.649157 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:43.651830 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:43.675083 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec7be9-6ebe-48fc-be64-70cb3f92dc7c None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:43.682020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:43.684205 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:43.684656 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:43.684871 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] PATH_INFO: `/v3/projects/ad31f1468e9c41eaa4bfec667e605b68` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:43.687165 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad31f1468e9c41eaa4bfec667e605b68)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:43.690834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:43.758693 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad31f1468e9c41eaa4bfec667e605b68', 'request_id': 'req-f87b713e-140e-4995-8392-e828a79b065a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:43.765499 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad31f1468e9c41eaa4bfec667e605b68', 'request_id': 'req-f87b713e-140e-4995-8392-e828a79b065a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:43.774413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad31f1468e9c41eaa4bfec667e605b68', 'request_id': 'req-f87b713e-140e-4995-8392-e828a79b065a'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:43.784557 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:43.790627 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] The token cache is being invalidate because project ad31f1468e9c41eaa4bfec667e605b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80184) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:43:43.791092 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:43.792174 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f87b713e-140e-4995-8392-e828a79b065a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:43:43.794278 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1370/2745] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:43:43 2022] DELETE /identity/v3/projects/ad31f1468e9c41eaa4bfec667e605b68 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:43:48.977239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:43:48.978719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:49.071063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b7aa4-7b5b-4fa2-962b-d35640fb86b3 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:43:49.089703 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:43:49.136278 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:43:49.136962 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:49.137245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:49.137526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:49.139327 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:49.150331 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:43:49.152087 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:43:49.161835 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1376/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:43:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:43:57.926388 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8677-d622-4853-97d3-a8f05a0e1fc6 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:43:57.927136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8677-d622-4853-97d3-a8f05a0e1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:43:57.927136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8677-d622-4853-97d3-a8f05a0e1fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:43:57.944944 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-12ea8677-d622-4853-97d3-a8f05a0e1fc6 None None] MFA Rules not processed for user `ac343929fc8542439bc627a1f445b1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:43:57.946170 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-12ea8677-d622-4853-97d3-a8f05a0e1fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:43:58.000937 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1371/2747] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:43:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:43:59.157693 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=22 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:44:07.993914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=25 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:44:15.613788 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:44:15.615221 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:44:15.706006 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fed833-92cb-4734-8c83-2b3bae6a9de3 service placement] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:44:15.709750 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:44:15.710395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:44:15.710678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:44:15.710953 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:44:15.712782 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:44:15.782949 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:44:15.784357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d04e1-1b71-4802-8d95-d77bd47c4893 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:44:15.794414 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1377/2748] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:44:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:44:25.789861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=59 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:44:32.139224 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb5d7-46c3-4787-a527-cf9b90784a08 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:44:32.139780 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb5d7-46c3-4787-a527-cf9b90784a08 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:44:32.139890 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eb5d7-46c3-4787-a527-cf9b90784a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:44:32.158631 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-715eb5d7-46c3-4787-a527-cf9b90784a08 None None] MFA Rules not processed for user `ae56ce45fd2d4acab2c1959a45dfe00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:44:32.159874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-715eb5d7-46c3-4787-a527-cf9b90784a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:44:32.216744 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1372/2749] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:44:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:44:42.206481 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=25 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:45:11.192604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.193218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.193218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] PATH_INFO: `/` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.194564 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1378/2750] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:45:11.201250 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69271bf-23a4-4bab-9948-ea1d78f8c034 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.201743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69271bf-23a4-4bab-9948-ea1d78f8c034 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.201935 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69271bf-23a4-4bab-9948-ea1d78f8c034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.244523 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-a69271bf-23a4-4bab-9948-ea1d78f8c034 None None] MFA Rules not processed for user `24385618d07d482d9567fe2602fa832a`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:45:11.245190 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-a69271bf-23a4-4bab-9948-ea1d78f8c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:45:11.252975 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1373/2751] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:45:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:45:11.270622 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:45:11.272358 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:45:11.290862 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f302d6-a90f-4dac-81f0-d68fa09dbb55 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:45:11.294690 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:45:11.295362 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.295652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.295930 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.303347 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:45:11.304780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b1cb-1129-4f09-9a4b-0e677b95fec4 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:45:11.313236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1379/2752] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:45:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:45:11.395279 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129322a2-b4cb-49c1-81af-11d2cd48ea08 None None] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.395593 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129322a2-b4cb-49c1-81af-11d2cd48ea08 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.395867 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129322a2-b4cb-49c1-81af-11d2cd48ea08 None None] PATH_INFO: `/` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.397265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1374/2753] 10.222.0.18 () {54 vars in 921 bytes} [Wed Jul 20 09:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:45:11.402860 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642741-bff0-4369-9dde-b3789118d7a6 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.403145 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642741-bff0-4369-9dde-b3789118d7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.403456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642741-bff0-4369-9dde-b3789118d7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.422050 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-2c642741-bff0-4369-9dde-b3789118d7a6 None None] MFA Rules not processed for user `0364cf432f5b44a9a62f08c4243fff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:45:11.423243 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-2c642741-bff0-4369-9dde-b3789118d7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:45:11.488507 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1380/2754] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Jul 20 09:45:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:45:11.499618 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129322a2-b4cb-49c1-81af-11d2cd48ea08 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:45:11.510429 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129322a2-b4cb-49c1-81af-11d2cd48ea08 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:45:11.514910 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:45:11.515609 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:11.515894 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:11.516171 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:11.526221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:45:11.528276 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:45:11.539166 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1375/2755] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:45:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:45:21.478915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=37 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:45:21.534204 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=17 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:45:33.020981 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:45:33.021469 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:45:33.021567 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:45:33.039508 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] MFA Rules not processed for user `ae56ce45fd2d4acab2c1959a45dfe00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:45:33.040700 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:45:33.047884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1381/2756] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:45:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:45:43.034281 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:46:14.939582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:14.948189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426aec40-7572-480a-a6d7-ce9cf7a68c62 service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:14.951807 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:14.952439 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:14.952741 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:14.953014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:14.954737 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:14.968696 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:14.970168 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd47d7-de21-4622-b8e8-dfadacbf2cd8 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:14.982158 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1376/2757] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:16.156266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:16.166504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25974d8-caa6-43f3-bbf0-564144ce5f8e None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:16.170488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:16.171127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:16.171441 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:16.171714 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:16.173464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:16.187317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:16.188739 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:16.199781 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1382/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:18.484006 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ec376-31ac-438f-9cdd-0f2fd5c3cf91 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:18.484498 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ec376-31ac-438f-9cdd-0f2fd5c3cf91 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:18.484604 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ec376-31ac-438f-9cdd-0f2fd5c3cf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:18.501954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-626ec376-31ac-438f-9cdd-0f2fd5c3cf91 None None] MFA Rules not processed for user `ac343929fc8542439bc627a1f445b1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:46:18.503832 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-626ec376-31ac-438f-9cdd-0f2fd5c3cf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:18.510808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1377/2759] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:46:26.194828 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=6 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:46:28.496671 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=9 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:46:36.868629 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:36.879778 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565527e-ea29-4942-bd4f-e74ecce977a7 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:36.884147 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:36.884836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:36.885119 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:36.885396 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:36.887420 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:36.899018 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:36.901390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:36.912723 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1383/2760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:38.222137 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:38.222628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:38.223253 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:38.241044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] MFA Rules not processed for user `ae56ce45fd2d4acab2c1959a45dfe00b`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:46:38.242249 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:38.249596 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1378/2761] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:46:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:46:42.498786 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:42.501193 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:42.529516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4754a-7d36-4dd6-bdaf-a114705c8c20 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:42.549648 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:42.550534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:42.551612 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:42.552195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:42.554424 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:42.569061 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:42.570849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:42.584532 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1384/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:46.978603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:46.980162 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:46.995386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6734b69-0cb3-4cd7-b851-ed6076fa0fd2 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:47.000634 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:47.001379 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:47.001740 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:47.002091 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:47.004148 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:47.017136 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:47.018827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:47.030199 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1379/2763] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:50.449838 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:50.452263 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:50.468628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73659d-6be0-48aa-b871-86200d814169 service nova] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:50.480529 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:50.481216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:50.481535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:50.481884 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:50.483854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:50.501525 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:50.503774 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:50.521345 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1385/2764] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:46:50.533622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:50.536178 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:50.602016 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f09401-e87d-48e0-be88-8b1d4c34dd2c service placement] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:50.606872 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:50.607833 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:50.608121 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:50.608399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] PATH_INFO: `/v3/users/0821d7c651dc4571834b8f3c40b225a9` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:50.609603 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0821d7c651dc4571834b8f3c40b225a9)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:50.611562 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:50.657705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0821d7c651dc4571834b8f3c40b225a9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:50.664494 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0821d7c651dc4571834b8f3c40b225a9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:50.665232 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0821d7c651dc4571834b8f3c40b225a9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:50.678133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0821d7c651dc4571834b8f3c40b225a9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:50.684817 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0821d7c651dc4571834b8f3c40b225a9'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:50.690730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1380/2765] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:46:50 2022] DELETE /identity/v3/users/0821d7c651dc4571834b8f3c40b225a9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:46:50.866472 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:50.877315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeedaa74-0d2e-4761-91b3-a053c0b9003e service placement] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:50.943456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:50.944351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:50.944626 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:50.945366 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] PATH_INFO: `/v3/projects/ee1753001c624ae3941cfc51b3d9d6ef` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:50.946254 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1753001c624ae3941cfc51b3d9d6ef)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:50.948239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:50.998804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1753001c624ae3941cfc51b3d9d6ef', 'request_id': 'req-3971929d-ae41-4fe2-a06b-7f786b49e9c7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.007992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1753001c624ae3941cfc51b3d9d6ef', 'request_id': 'req-3971929d-ae41-4fe2-a06b-7f786b49e9c7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.015672 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1753001c624ae3941cfc51b3d9d6ef', 'request_id': 'req-3971929d-ae41-4fe2-a06b-7f786b49e9c7'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.031565 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] The token cache is being invalidate because project ee1753001c624ae3941cfc51b3d9d6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:46:51.031847 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.032678 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.034216 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1386/2766] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:46:50 2022] DELETE /identity/v3/projects/ee1753001c624ae3941cfc51b3d9d6ef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:46:51.038874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:51.040307 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:51.055385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80aac75-f3bd-4a1c-9104-b5f4c20b4f3b admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:51.119340 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:51.120227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:51.120506 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:51.120774 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] PATH_INFO: `/v3/users/b6ad8843675246bcba9b714502e63bf8` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:51.121954 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ad8843675246bcba9b714502e63bf8)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:51.123962 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:51.125378 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:46:51.171550 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ad8843675246bcba9b714502e63bf8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.179341 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ad8843675246bcba9b714502e63bf8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.179893 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ad8843675246bcba9b714502e63bf8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.191914 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ad8843675246bcba9b714502e63bf8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.198570 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ad8843675246bcba9b714502e63bf8'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.204424 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1381/2767] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:46:51 2022] DELETE /identity/v3/users/b6ad8843675246bcba9b714502e63bf8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:46:51.379851 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:46:51.381341 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:46:51.394191 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971929d-ae41-4fe2-a06b-7f786b49e9c7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:46:51.443535 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:46:51.454610 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:46:51.455499 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:46:51.455785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:46:51.456063 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] PATH_INFO: `/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:46:51.457964 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ef7546dc454ebaa3a3ef9d2f11e36c)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:46:51.460027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:46:51.505466 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ef7546dc454ebaa3a3ef9d2f11e36c', 'request_id': 'req-a29fe5bf-71e4-4b25-878e-1e4193d708d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.512480 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ef7546dc454ebaa3a3ef9d2f11e36c', 'request_id': 'req-a29fe5bf-71e4-4b25-878e-1e4193d708d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.517943 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ef7546dc454ebaa3a3ef9d2f11e36c', 'request_id': 'req-a29fe5bf-71e4-4b25-878e-1e4193d708d2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.529344 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] The token cache is being invalidate because project 41ef7546dc454ebaa3a3ef9d2f11e36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:46:51.529628 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.530264 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:46:51.531857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1387/2768] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:46:51 2022] DELETE /identity/v3/projects/41ef7546dc454ebaa3a3ef9d2f11e36c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:01.201014 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:01.526323 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:51.844630 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:51.846134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:51.859399 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901c9e9-29e2-45a6-920b-79ddd26a78c9 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:51.914811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:51.915730 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:51.916018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:51.916293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] PATH_INFO: `/v3/users/ae56ce45fd2d4acab2c1959a45dfe00b` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:51.917643 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae56ce45fd2d4acab2c1959a45dfe00b)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:51.919610 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:51.966712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae56ce45fd2d4acab2c1959a45dfe00b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:51.974018 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae56ce45fd2d4acab2c1959a45dfe00b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:51.974837 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae56ce45fd2d4acab2c1959a45dfe00b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:51.986582 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae56ce45fd2d4acab2c1959a45dfe00b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:51.993263 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-60879a08-00be-43fd-96c5-07f1207f4634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae56ce45fd2d4acab2c1959a45dfe00b'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:51.999739 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1382/2769] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:47:51 2022] DELETE /identity/v3/users/ae56ce45fd2d4acab2c1959a45dfe00b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:52.172069 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.173536 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:52.187833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe5bf-71e4-4b25-878e-1e4193d708d2 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.247655 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.248521 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.248836 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.249115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] PATH_INFO: `/v3/projects/a1fe308758354ddfb050def8345355ff` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.250325 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fe308758354ddfb050def8345355ff)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.252352 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.299849 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fe308758354ddfb050def8345355ff', 'request_id': 'req-471945af-f192-426d-925d-e1206532a9f2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:52.306745 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fe308758354ddfb050def8345355ff', 'request_id': 'req-471945af-f192-426d-925d-e1206532a9f2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:52.312516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fe308758354ddfb050def8345355ff', 'request_id': 'req-471945af-f192-426d-925d-e1206532a9f2'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:52.324173 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] The token cache is being invalidate because project a1fe308758354ddfb050def8345355ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:47:52.324460 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:52.325092 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-471945af-f192-426d-925d-e1206532a9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:52.326631 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1388/2770] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:47:52 2022] DELETE /identity/v3/projects/a1fe308758354ddfb050def8345355ff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:52.331640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.331945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.332223 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.370284 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:52.371541 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:52.438058 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1383/2771] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:52.445421 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.445716 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.446020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.465301 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:52.466504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:52.474540 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1389/2772] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:47:52.478706 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.484755 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:52.488678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a522-5118-4d5b-abdd-ab501a546653 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.494477 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.495445 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.496019 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.496356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.497306 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:47:52.497621 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.499207 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.511255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1384/2773] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:47:52.515571 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.524649 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde95ac-f696-444c-aa9d-d3e06ca2c9d8 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.529918 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.530555 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.530833 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.531109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.531834 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.533718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.568029 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1390/2774] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:47:52 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:47:52.571886 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.582390 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec1541-7aa9-4c49-8b0a-0fa751516c5f admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.638197 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.638854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.639134 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.639440 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.640131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.642030 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.664903 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1385/2775] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:47:52 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:47:52.666025 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.676988 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e008e8-8733-479c-a463-84d93ea4987f admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.681558 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.682208 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.683082 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.683630 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.684361 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.685940 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.696319 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1391/2776] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:47:52.700978 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.711090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7565f5-c250-4bcd-9545-c288b5818bfb admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.715908 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.716565 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.716862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.717125 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.717838 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.719460 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804eded8-45bf-4f16-b32d-80bfc9349dca admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.731227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1386/2777] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:47:52.735357 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.746074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70670d95-f362-4627-85d9-26ea1aa4e3bf admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.750652 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.751552 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.751830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.752100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] PATH_INFO: `/v3/projects/383efe1b3a2b4d379d0d5671c3b6ace2/users/5c8b872e82c54f7591c8dd3705da1aff/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.767088 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=383efe1b3a2b4d379d0d5671c3b6ace2, user_id=5c8b872e82c54f7591c8dd3705da1aff, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.769742 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.782114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1392/2778] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:47:52 2022] PUT /identity/v3/projects/383efe1b3a2b4d379d0d5671c3b6ace2/users/5c8b872e82c54f7591c8dd3705da1aff/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:52.792035 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.792526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.792808 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.819742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] MFA Rules not processed for user `5c8b872e82c54f7591c8dd3705da1aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:52.820958 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:52.884919 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1387/2779] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:52.895798 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:52.897516 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:52.918768 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:52.972660 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecc06e-3f3c-4975-9452-14bba45f0f1d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:52.978152 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:52.978799 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:52.979084 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:52.979935 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:52.987751 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:52.989335 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b58418-affe-4044-84ae-7d3edaeb3570 service nova] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:52.998256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1393/2780] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:47:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:47:53.205873 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:53.267816 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5f98-1676-494b-9b67-198e79e6a8fc None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:53.272348 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:53.273227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:53.273509 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:53.273789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] PATH_INFO: `/v3/users/5c8b872e82c54f7591c8dd3705da1aff` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:53.274980 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8b872e82c54f7591c8dd3705da1aff)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:53.277602 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:53.279087 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:53.326827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8b872e82c54f7591c8dd3705da1aff'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.333862 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8b872e82c54f7591c8dd3705da1aff'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.334512 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8b872e82c54f7591c8dd3705da1aff'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.347846 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8b872e82c54f7591c8dd3705da1aff'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.354646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8b872e82c54f7591c8dd3705da1aff'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.360584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1388/2781] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:47:53 2022] DELETE /identity/v3/users/5c8b872e82c54f7591c8dd3705da1aff => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:53.365075 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:53.365534 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:53.365647 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:53.382704 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:53.383919 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:53.446654 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1394/2782] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:53.456827 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:53.458236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:53.515021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f8f80-edbd-4c8b-a7f8-986ce5eef40d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:53.518685 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:53.519353 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:53.519641 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:53.519918 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:53.529255 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:53.530694 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031f3b8-d98d-41be-9af0-b5b75b9fc096 service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:53.540203 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1389/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:47:53.805954 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:53.807994 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:53.820183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fc864-a9b7-4114-80d0-a00a67e94fec None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:53.825258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:53.826133 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:53.826412 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:53.826683 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] PATH_INFO: `/v3/projects/383efe1b3a2b4d379d0d5671c3b6ace2` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:53.827904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=383efe1b3a2b4d379d0d5671c3b6ace2)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:53.829869 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:53.878488 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383efe1b3a2b4d379d0d5671c3b6ace2', 'request_id': 'req-e762dc1b-b216-40fc-b094-178bbe063e7d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.886230 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383efe1b3a2b4d379d0d5671c3b6ace2', 'request_id': 'req-e762dc1b-b216-40fc-b094-178bbe063e7d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.891879 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383efe1b3a2b4d379d0d5671c3b6ace2', 'request_id': 'req-e762dc1b-b216-40fc-b094-178bbe063e7d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.906132 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] The token cache is being invalidate because project 383efe1b3a2b4d379d0d5671c3b6ace2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:47:53.906418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.907056 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-e762dc1b-b216-40fc-b094-178bbe063e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:47:53.908741 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1395/2784] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:47:53 2022] DELETE /identity/v3/projects/383efe1b3a2b4d379d0d5671c3b6ace2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:53.913611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:53.914110 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:53.914227 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:53.933308 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:53.934507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:53.993154 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1390/2785] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:54.002504 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.002796 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.003100 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.021498 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:54.022692 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:54.031464 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1396/2786] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:54.035622 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.045819 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863f73-6ed2-4967-a1aa-2d924d7b40aa None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.050323 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.051045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.051358 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.051637 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.052351 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:47:54.052655 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.054220 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.066135 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1391/2787] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:47:54.070241 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.079909 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf05ee-1624-4b0e-a640-9fb2ec5675de None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.084584 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.085223 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.085503 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.085775 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.086450 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.088376 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.107941 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:54.116218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1397/2788] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:47:54.120869 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.130811 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba391694-da5a-4248-a6e8-0c0c665c0f2d admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.135678 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:54.189534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.190122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.190408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.190688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.191401 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.193312 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.212293 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1392/2789] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/users => generated 278 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:47:54.217073 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.227473 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509e481-8386-4319-9f61-7a2deaf27af6 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.234058 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.234718 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.235003 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.235274 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.236027 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.237680 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.248576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1398/2790] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:47:54.252584 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.263273 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bb98-84a9-4341-9cdf-7b54d53dd52c admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.267882 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.268536 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.268825 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.269099 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.269851 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.271480 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0cdd-c83e-4cbb-a33a-3f8d3a6ad94a admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.282200 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1393/2791] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:47:54.285928 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.297342 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8f416-e9ca-4b1a-8fd8-5a59605c5284 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.301915 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.302773 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.303049 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.303346 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] PATH_INFO: `/v3/projects/a6985abafdaa413392ed70b383387780/users/9328eb023d0f414da6e761635b950953/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.317927 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6985abafdaa413392ed70b383387780, user_id=9328eb023d0f414da6e761635b950953, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.320579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.333871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1399/2792] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:47:54 2022] PUT /identity/v3/projects/a6985abafdaa413392ed70b383387780/users/9328eb023d0f414da6e761635b950953/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:54.345790 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.346079 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.346356 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.372645 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] MFA Rules not processed for user `9328eb023d0f414da6e761635b950953`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:54.373841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:54.437966 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1394/2793] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:47:54.442333 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.504122 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d155d-6a8c-41cd-a009-5a19faeed303 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.508777 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.509442 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.509719 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.509992 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.511317 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.513215 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.537804 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1400/2794] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/projects => generated 357 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:47:54.541520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.553010 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23d958-80a5-4dee-becd-73de90784104 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.608526 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.609189 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.609472 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.609745 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.610420 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.612382 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.632040 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1395/2795] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:47:54.636588 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.646787 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4aac-99f3-4314-8e3f-dfe24d70a8ea admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.651252 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.651936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.652214 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.652486 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.653195 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.654913 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.665258 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1401/2796] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:47:54.670032 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.680112 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5042d3-af55-400f-875b-e652ab5c3664 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.685588 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.686520 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.686854 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.687133 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] PATH_INFO: `/v3/projects/330a139f6c044b849ffa54f94a4f7008/users/9f482ea5817e41dba1a42ee9851dc384/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.689385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:54.703533 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=330a139f6c044b849ffa54f94a4f7008, user_id=9f482ea5817e41dba1a42ee9851dc384, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.706090 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.718982 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1396/2797] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:47:54 2022] PUT /identity/v3/projects/330a139f6c044b849ffa54f94a4f7008/users/9f482ea5817e41dba1a42ee9851dc384/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:54.723045 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.733411 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaacecc9-5f44-468a-9d44-efec95b85f7c admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.789522 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.790183 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.790463 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.790733 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.791471 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.793901 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58891e9-d56f-491e-8f79-e2397dd472f4 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.804282 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1402/2798] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:47:54.808721 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.818858 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e555-dc84-48da-ab4e-d25098fdfc00 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:54.823587 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:54.824482 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.824767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.825045 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] PATH_INFO: `/v3/projects/330a139f6c044b849ffa54f94a4f7008/users/9f482ea5817e41dba1a42ee9851dc384/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.827062 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=330a139f6c044b849ffa54f94a4f7008, user_id=9f482ea5817e41dba1a42ee9851dc384, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:54.829727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:54.840899 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1397/2799] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:47:54 2022] PUT /identity/v3/projects/330a139f6c044b849ffa54f94a4f7008/users/9f482ea5817e41dba1a42ee9851dc384/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:47:54.850780 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:54.851074 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:54.851390 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:54.877904 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] MFA Rules not processed for user `9f482ea5817e41dba1a42ee9851dc384`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:47:54.879104 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:54.898886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:47:54.948738 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1403/2800] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:47:54.959809 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:54.961561 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:47:55.040044 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeafca-1dc9-4572-a20a-997be324acdd admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:55.044322 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:55.044965 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:55.045502 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:55.045806 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:55.054700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:55.056152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28d6d5-76c9-484b-98d7-aaf982c1552b service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:55.065611 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1398/2801] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:47:55.135237 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:47:55.145117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82931df2-1911-4496-b4e4-f01eb5cecae5 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:47:55.148882 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:47:55.149530 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:47:55.149815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:47:55.150204 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:47:55.206048 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:47:55.207439 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:47:55.217002 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1404/2802] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:47:55 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:48:05.061446 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:05.212222 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=47 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:11.295001 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:11.295542 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:11.295648 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:11.313101 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] MFA Rules not processed for user `ac343929fc8542439bc627a1f445b1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:11.314302 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:11.367911 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1399/2803] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:48:11.686407 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:11.687885 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:11.744297 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7de2be-238e-437c-aab8-7ee64b4f847f service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:11.748174 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:11.748827 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:11.749108 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:11.749382 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:11.751177 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:11.762109 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:11.763639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:11.772920 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1405/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:48:12.204214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:12.244353 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:12.245725 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:12.283969 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd407795-03f5-4d0b-96aa-f8729ec6fde0 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:12.289844 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:12.290664 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:12.290949 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:12.291226 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] PATH_INFO: `/v3/users/9328eb023d0f414da6e761635b950953` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:12.292486 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9328eb023d0f414da6e761635b950953)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:12.294994 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:12.329117 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeff849-12b1-458c-9d64-039591d35b09 service neutron] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:12.334415 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:12.335245 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:12.335639 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:12.335857 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:12.338239 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:12.351115 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:12.352763 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e0b63-59b9-4232-b3d3-8de04bb48158 service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:12.359640 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9328eb023d0f414da6e761635b950953'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.363151 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1406/2805] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:48:12.367501 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9328eb023d0f414da6e761635b950953'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.368251 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9328eb023d0f414da6e761635b950953'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.381061 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9328eb023d0f414da6e761635b950953'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.388519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9328eb023d0f414da6e761635b950953'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.394761 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1400/2806] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:48:12 2022] DELETE /identity/v3/users/9328eb023d0f414da6e761635b950953 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:12.399315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:12.399618 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:12.399895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:12.424813 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:12.426036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:12.492854 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1407/2807] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:48:12.503628 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:12.505234 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:12.536812 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:12.570646 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b3aa-206e-4742-92f3-146796303152 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:12.574386 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:12.575225 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:12.575574 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:12.575849 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:12.585951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:12.587519 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:12.598262 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1401/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:48:12.882891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:12.884395 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:12.896791 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f900205-f877-46a8-92aa-ea47cd818fa1 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:12.901807 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:12.903251 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:12.903573 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:12.903830 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] PATH_INFO: `/v3/projects/a6985abafdaa413392ed70b383387780` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:12.905036 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6985abafdaa413392ed70b383387780)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:12.906975 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:12.941479 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:12.958167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6985abafdaa413392ed70b383387780', 'request_id': 'req-c96df302-2f9a-4dae-8767-1fdc27b9f99d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.965790 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6985abafdaa413392ed70b383387780', 'request_id': 'req-c96df302-2f9a-4dae-8767-1fdc27b9f99d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.971236 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6985abafdaa413392ed70b383387780', 'request_id': 'req-c96df302-2f9a-4dae-8767-1fdc27b9f99d'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.982635 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] The token cache is being invalidate because project a6985abafdaa413392ed70b383387780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:48:12.982914 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.983566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:12.985356 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1408/2809] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:48:12 2022] DELETE /identity/v3/projects/a6985abafdaa413392ed70b383387780 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:12.989160 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:12.998767 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5a0c2-899b-45d7-ba8b-b844216ab60f service neutron] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:13.053831 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:13.054747 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.055048 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.055359 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] PATH_INFO: `/v3/users/9f482ea5817e41dba1a42ee9851dc384` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.056547 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f482ea5817e41dba1a42ee9851dc384)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:13.058470 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:13.103389 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f482ea5817e41dba1a42ee9851dc384'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.111009 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f482ea5817e41dba1a42ee9851dc384'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.111647 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f482ea5817e41dba1a42ee9851dc384'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.123505 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f482ea5817e41dba1a42ee9851dc384'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.130221 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.notifications [None req-20b62079-6c9e-4e02-a95a-c7f36e0df5bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f482ea5817e41dba1a42ee9851dc384'} {{(pid=80184) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.136131 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1402/2810] 10.222.0.18 () {58 vars in 1223 bytes} [Wed Jul 20 09:48:12 2022] DELETE /identity/v3/users/9f482ea5817e41dba1a42ee9851dc384 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:13.505523 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:13.507057 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:13.526253 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96df302-2f9a-4dae-8767-1fdc27b9f99d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:13.590248 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:13.591120 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.591435 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.591705 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] PATH_INFO: `/v3/projects/330a139f6c044b849ffa54f94a4f7008` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.592911 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=330a139f6c044b849ffa54f94a4f7008)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:13.594896 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:13.642288 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330a139f6c044b849ffa54f94a4f7008', 'request_id': 'req-ae87cd12-e27a-46ef-b507-f06cd5357d95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.651351 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330a139f6c044b849ffa54f94a4f7008', 'request_id': 'req-ae87cd12-e27a-46ef-b507-f06cd5357d95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.658853 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330a139f6c044b849ffa54f94a4f7008', 'request_id': 'req-ae87cd12-e27a-46ef-b507-f06cd5357d95'} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.671699 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] The token cache is being invalidate because project 330a139f6c044b849ffa54f94a4f7008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80185) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:48:13.671991 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.672658 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.notifications [None req-ae87cd12-e27a-46ef-b507-f06cd5357d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80185) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:48:13.674300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1409/2811] 10.222.0.18 () {58 vars in 1232 bytes} [Wed Jul 20 09:48:13 2022] DELETE /identity/v3/projects/330a139f6c044b849ffa54f94a4f7008 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:13.679789 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.680336 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.680534 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.699727 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:13.701039 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:13.751051 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:13.775444 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1403/2812] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:48:13.785226 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.785582 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.785812 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.808159 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] MFA Rules not processed for user `cee8d1e7eb98438287c0fee8c54bc3cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:13.809691 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:13.818852 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1410/2813] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:48:13.822863 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:13.832865 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8692e1b-083e-49d6-beff-ba6b81631431 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:13.837810 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:13.838599 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.838889 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.839271 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.840874 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80184) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:48:13.841750 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:13.843732 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:13.862384 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1404/2814] 10.222.0.18 () {58 vars in 1152 bytes} [Wed Jul 20 09:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:48:13.864004 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:13.875078 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e591c-c5a5-47a6-b3d7-ba086ab4b342 None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:13.881495 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:13.882256 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.882603 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.883001 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.883785 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:13.885748 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:13.915170 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1411/2815] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:48:13 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:48:13.920373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:13.932373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b15b6-8f7a-45f7-baf2-b22aef4aa483 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:13.990291 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:13.991456 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:13.991742 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:13.992017 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:13.992704 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:13.994700 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.016473 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1405/2816] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:48:13 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:48:14.020566 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.032662 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dec1b-5a2e-4f46-8b22-159a3ae26e76 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.040218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.040875 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.041158 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.041456 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.042180 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.043861 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.056285 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1412/2817] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:48:14.060295 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.071240 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3264e5-2b41-4181-b12e-4d9b4dc1f985 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.079113 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.079845 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.080130 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.080408 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.081144 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.082952 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24657cb-5850-48b1-ba78-cc4930143b43 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.095760 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1406/2818] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:48:14.099977 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.110604 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58737d4-f229-488f-a3dd-892231af0d6e admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.115579 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.116475 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.116752 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.117167 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] PATH_INFO: `/v3/projects/1d3462f9bdb74b448b958f2c91ca9bbc/users/54ad6f1d0afd458fb774299c2088010c/roles/011a188e9a5b4469804831444edf3f23` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.119895 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:14.141275 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3462f9bdb74b448b958f2c91ca9bbc, user_id=54ad6f1d0afd458fb774299c2088010c, role_id=011a188e9a5b4469804831444edf3f23)` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.144070 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.160114 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1413/2819] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:48:14 2022] PUT /identity/v3/projects/1d3462f9bdb74b448b958f2c91ca9bbc/users/54ad6f1d0afd458fb774299c2088010c/roles/011a188e9a5b4469804831444edf3f23 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:48:14.170578 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.171370 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.171781 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.202743 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.auth.core [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] MFA Rules not processed for user `54ad6f1d0afd458fb774299c2088010c`. Rule list: `[]` (Enabled: `True`). {{(pid=80184) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:14.204186 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:14.278385 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1407/2820] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:48:14.283321 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.352217 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f11c2-55c9-4788-95bf-a29adab3bef7 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.357015 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.357681 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.357972 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.358246 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.358938 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.360878 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.388886 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1414/2821] 10.222.0.18 () {60 vars in 1152 bytes} [Wed Jul 20 09:48:14 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:48:14.393435 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.405835 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9015cd-5ba7-46fc-a7e5-ab714cae32c5 None None] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.451265 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:14.469528 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.470214 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.470500 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.470778 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.471517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.473776 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.501241 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1408/2822] 10.222.0.18 () {60 vars in 1143 bytes} [Wed Jul 20 09:48:14 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:48:14.502418 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.514020 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d41b66-1f9c-4cb1-9833-81a7a0791b16 admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.518811 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.519511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.519803 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.520089 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.520810 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.522554 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.540131 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1415/2823] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:48:14.541330 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.553948 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e43f1-f2b9-4377-a647-31f62067cea4 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.564668 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.565786 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.566097 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.566373 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] PATH_INFO: `/v3/projects/e93b0e0994904180907198a4b6df7b9d/users/0531e73d4f4543bba31275f553ff2ba6/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.585834 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93b0e0994904180907198a4b6df7b9d, user_id=0531e73d4f4543bba31275f553ff2ba6, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.588563 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.610798 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1409/2824] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:48:14 2022] PUT /identity/v3/projects/e93b0e0994904180907198a4b6df7b9d/users/0531e73d4f4543bba31275f553ff2ba6/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:14.614872 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.627986 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453f36d-b2a6-4751-83d6-3502821ff66d admin admin] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.711891 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.712889 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.713576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.713576 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.714168 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.716184 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f67f54-4e1b-49cb-8de5-bb2d5835cce1 admin admin] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.731506 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1416/2825] 10.222.0.18 () {58 vars in 1121 bytes} [Wed Jul 20 09:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:48:14.735517 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.748841 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88cff7-bf53-4012-892e-8c920638b328 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:14.755493 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee8d1e7eb98438287c0fee8c54bc3cc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c93172815ff14813957f03bf548f6155', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:14.756688 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.757194 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.757712 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] PATH_INFO: `/v3/projects/e93b0e0994904180907198a4b6df7b9d/users/0531e73d4f4543bba31275f553ff2ba6/roles/743f7cb41c634460b2981782f093ad19` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.760106 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93b0e0994904180907198a4b6df7b9d, user_id=0531e73d4f4543bba31275f553ff2ba6, role_id=743f7cb41c634460b2981782f093ad19)` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:14.763268 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:14.775021 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1410/2826] 10.222.0.18 () {60 vars in 1482 bytes} [Wed Jul 20 09:48:14 2022] PUT /identity/v3/projects/e93b0e0994904180907198a4b6df7b9d/users/0531e73d4f4543bba31275f553ff2ba6/roles/743f7cb41c634460b2981782f093ad19 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:48:14.845663 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] REQUEST_METHOD: `POST` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:14.845985 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:14.846228 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:14.875511 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.auth.core [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] MFA Rules not processed for user `0531e73d4f4543bba31275f553ff2ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80185) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:48:14.876815 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:14.920802 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:14.950240 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1417/2827] 10.222.0.18 () {58 vars in 957 bytes} [Wed Jul 20 09:48:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:48:14.961705 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:14.963590 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:15.026488 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db2969-3c57-4597-a01c-41bfe2b4c162 admin admin] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:15.031364 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:15.032028 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:15.032314 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:15.032594 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:15.042026 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:15.043674 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:15.053946 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1411/2828] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:48:15.129066 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:15.139674 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25391644-743d-4216-b1c3-c93ea72ce1bb None None] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:15.143266 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '267cd2f0b4d442ee9ab2ee668394472c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:15.143936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:15.144218 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:15.144491 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:15.202708 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:15.204200 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:15.213307 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1418/2829] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:48:25.049236 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=51,keystone:INSERT=5 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:25.209127 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=38 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:25.470452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:25.472256 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:25.549152 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4848cc-fb72-461d-8fcb-8654bbfa29d7 service cinder] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:25.554713 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0364cf432f5b44a9a62f08c4243fff5e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:25.555393 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:25.555679 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:25.555951 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:25.557720 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:25.570417 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:25.572020 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:25.582483 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1412/2830] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:48:35.578000 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=34 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:48:49.363841 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] Authenticating user token {{(pid=80185) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:48:49.366526 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:49.502083 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95589ee-d187-45c9-b95f-9f385d3c6205 service cinder] Validating token access rules against request {{(pid=80185) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:48:49.510300 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dddb4f1f2b443049a249f9ab3eeb23f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80185) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:48:49.511544 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] REQUEST_METHOD: `GET` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:48:49.511936 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] SCRIPT_NAME: `/identity` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:48:49.512392 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80185) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:48:49.515315 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.fernet_utils [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80185) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:48:49.535871 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:48:49.538562 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe053f0-f5fb-403c-a352-5f354cb74517 service placement] RBAC: Authorization granted {{(pid=80185) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:48:49.557697 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: [pid: 80185|app: 0|req: 1419/2831] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:48:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:48:59.549729 dv-cinder-822822-3-iscsi devstack@keystone.service[80185]: DEBUG dbcounter [-] [80185] Writing DB stats keystone:SELECT=32 {{(pid=80185) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:52:03.705788 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] Authenticating user token {{(pid=80184) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:52:03.707122 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:52:03.764818 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3f55f-af0e-4e72-8652-e938bd405f94 service nova] Validating token access rules against request {{(pid=80184) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:52:03.768450 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24385618d07d482d9567fe2602fa832a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84bb5538f2344f0aaf64d7e0b22d4a0a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80184) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:52:03.769105 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] REQUEST_METHOD: `GET` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:52:03.769452 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] SCRIPT_NAME: `/identity` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:52:03.769736 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80184) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:52:03.771560 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.fernet_utils [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80184) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:52:03.782945 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:52:03.784413 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc5e45-09a4-41a2-81c9-9dc3da92a15c service neutron] RBAC: Authorization granted {{(pid=80184) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:52:03.793507 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: [pid: 80184|app: 0|req: 1413/2832] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Jul 20 09:52:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:52:13.789151 dv-cinder-822822-3-iscsi devstack@keystone.service[80184]: DEBUG dbcounter [-] [80184] Writing DB stats keystone:SELECT=26 {{(pid=80184) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}